The following presentation was given at MWLUG 2015 in Atlanta Georgia. If you have any questions please let me know 🙂
This was very kindly recorded by my good friend David Navarre and rendering it to youtube did not apparently help his insomnia. Not sure what to read into that 🙂
In this article I will further discuss how tom improve the user experience of an automated partial Refresh on an user’s XPage. Although these posts were originally about using Bluemix to host the node.js server I kinda feel that the focus has drifted onto websockets more than Bluemix. So in an attempt to make it easier to find I am going to use the Websockets in XPages title moniker for a few posts and then go back to Bluemix 🙂
Introduction
In the last article we looked at how to push a automated partialRefresh to a XPage application using websockets. In that article it was noted that the user experience was not ideal because the whole panel refreshed without the user knowing about it. For some apps that is appropriate and for others it may not be. At this point in his career Dave Leedy is impressed he gave someone else and idea and I quote: “wow! that’s fricken awesome!!!”
So, that’s not a great user experience – what if they were doing something at the time?
Yes I was thinking that too! So I believe we can improve the user experience a little and take what Dave suggested and tweak it a little. Now where have a seen something which let’s the user know there is new data changes but doesn’t refresh the page without their action……….
oh yeah that.
Instead of refreshing the control automatically, we will make the message create a “refresh” icon on the page which the user can then update at their leisure.
The modified code is all in what happens when the page receives the refresh socket message. I added a jQuery rotate function just for some added “je ne sais quoi“. In the function we can see that when the refresh event is detected by the socket code the refreshControl function is called. This in turn makes the hidden refreshIcon visible, adds an onClick event and then rotates it. The onClick event performed the partialRefreshGet as we saw in the previous example turning the page briefly grey. We then hide the icon and remove the click event (to avoid piling on multiple events as the page gets continually refreshed)
// Function to add a message to the page
var refreshControl = function(data) {
$('.refreshIcon')
.css({display: 'block'})
.on('click', function(){
var temp = $('[id*='+data.refreshId+']').css({background: '#CCCCCC', color: 'white'}).attr('id')
XSP.partialRefreshGet(temp, {})
$(this).css({display: 'none'}).off('click')
})
.rotate({
angle:0,
animateTo:360,
easing: function (x,t,b,c,d){ // t: current time, b: begInnIng value, c: change In value, d: duration
return c*(t/d)+b;
}
})
};
// When a refresh message is received from the server
// action it
socket.on('refresh', function(data) { refreshControl(data); });
The following video quickly demonstrates the new capability.
Conclusion
In this brief article we concentrated on how to improve a user experience by notifying them that changes were pending and then allowing them to determine when the changes were made.
I still don’t think this is as optimal as I would like but you get the idea. As I said a long time ago – the more DOM you are reloading the worse the user experience. With a viewPanel we are kinda limited on what we can and cannot refresh. A better option may be to architect the application just the new data and update as appropriate……….
In this article I will demonstrate how using targeted websockets messages we are able to refresh user data on pertinent screens within an application, and keep user’s data up to date.
Introduction
In previous articles I have discussed the creation of a nodejs websockets service within Bluemix and how we are able to send messages to specific web pages using the socket.io rooms capability. Both of those examples were proofs of concept and the messages were generated in the browser via firebug console commands. We are going to look at how we can automate these messages and begin to build a user case for using websockets within our applications.
Disclaimer: This idea for an example was the brainchild of David Leedy and in many ways it is a genius example to relate websockets to XPages functionality. And at the same time, I am absolutely disgusted that I am even talking about this because I would never dream of actually implementing this within an application. The fact that the page changes without the user’s knowledge is poor, and the fact that it refreshes the entire control when only a small piece of data changes is horrible. All that said however, this is still a demonstration of the capability and in future blog posts on the subject I will actually show example which I would be proud to actually put into one of my own applications 😉
Keeping data up to data on an XPage
Let’s say we have a sample application with a simple XPages viewPanel on it
Somewhere within the application – someone else makes a change to the data
The only way to see the change would be to refresh the page – and you the user would never know when.
This can be pseudo-automated from a user experience perspective in a number of ways but they all involve periodic checking for updates on way or another. If you scale that over many users this is extremely inefficient.
That is where websockets comes in very nicely.
Pushing to specific rooms
In the previous article I demonstrated how to record an XPage as a “room” dynamically and in this example we will do the same thing.
var temp = (location.href.indexOf('copper.xomino.com')>0) ? "http://localhost:3000" : 'http://xominosocket.mybluemix.net'
var socket = io.connect(temp)
// Send message to server that user has joined
nickname=$('.username').text()
var xpageName = location.href.split('.nsf/')[1]
xpageName = xpageName.split('.xsp')[0]
socket.emit('joinRoom', xpageName, nickname);
Automating a partialRefresh
In a similar manner to listening for a new chat message and then acting upon it, we are going to listen for a “refresh” socket event and then action it. In this case we are also going to pass in the id of the XSP control we want to refresh. For the sake of this example I am also using some CSS to make the element appear momentarily grey (see the video and all will be clear)
// Function to add a message to the page
var refreshControl = function(data) {
//Get the refreshId from the incoming data and color the control grey
//then get the id of the element via the is attribute
var temp = $('[id*='+data.refreshId+']').css({background: '#CCCCCC', color: 'white'}).attr('id')
//With the known id - trigger a XPages partial refresh of the control
XSP.partialRefreshGet(temp, {})
};
// When a refresh message is received from the server
// action it by calling the refreshControl function
socket.on('refresh', function(data) { refreshControl(data); });
Creating a new listener on the node.js server
I created a new route to post my refresh data to “xpagesRefresh”. When data is POSTed at xpagesRefresh it is parsed and send back out via websockets using the “refresh” socket event.
// Handle the form POST containing the name and , reply with the language
app.post('/xpagesRefresh', cors(corsOptions), function(req, res){
var request_data = {};
if (req.body){
request_data = {
'refreshId': req.body.refreshId,
'nickname': req.body.nickname,
'rt': 'text'
};
}
var data = { refreshId: request_data.refreshId, nickname: request_data.nickname};
console.log("POSTing at xpagesRrefresh")
console.log(req.body)
io.sockets.emit("refresh", data);
res.send(data);
});
Automating the partial refresh via the user action
In a real application we are not going to have someone sitting on a browser pushing updates via firebug. We want to be able to create the update when the XPage is saved in the first place. To do this we transfer the code we saw in previous firebug blog example into the onComplete of a Save button. In this way when a document is updated within the application. A notification is sent out to all the people looking at the page, updating the data for them.
The video below shows how we are able to trigger a partial refresh after updating a document within the application. Note the screen flickers grey as the CSS change happens before the partial refresh. It is so quick it is almost instant.
Pushing to Bluemix
As before, pushing my new code to Bluemix is as simple as checking it in to the jazz hub repository and redeploying. In the picture below we can see that we are using marky.psclistens.com as the application domain and not copper.xomino.com. As we showed before, if not copper then use Bluemix.
We create the capability locally, tested it, proved it and deployed it seamlessly to Bluemix with a “Commit and Push” – and that is *so* cool 🙂
Conclusion
In this article we saw how to trigger core XPages functionality automatically without the user having to interact with the application. In the next article we will look at how to improve on this, frankly horrible, user experience.
In this article I will demonstrate how the use of socket.io “rooms” enables us to send message to users who are only accessing specific pages within our application, rather than blanket messages to all users.
Introduction
In the previous article I demonstrated how to use a message POST to the node.js server which could then be turned into a chat message and sent out to all users. While this is a nice example it only serves as such and does not have significant business value. In this article I will begin the peel back the potential for much greater application flexibility through the controlled use of targeted Websockets messages to users of an application.
Most applications have more than one “page” within it and we may wish to send a message to users of one page rather than all pages. Conversely we may want to only send messages to users who are viewing certain types of information wherever they are within the application.
In this article we are still going to use the chat example but this will be the last time we use “chat” as the use case. In future articles I will look into more practical applications of Websockets messaging within an application.
Using rooms within socket.io
Looking at the socket.io documentation for rooms and namespaces you can see that the API exposes the ability to create individual communication channels between the server and the application users.
How this translates to our application on the node.js server looks like this:
socket.on('joinRoom', function(room, name){
socket.join(room);
io.sockets.in(room).emit('notice', name+" has joined");
console.log(name+" has joined room "+room)
})
When the “joinRoom” event is registered on the server then the socket is joined to the room name which is passed in. A message is the broadcast specifically out to all the existing room members that the new user has joined. Note that this message is room specific because of the io.sockets.in(room) rather than a blanket message to all users.
With this code in place on any page we can register our application page (chat room in this case) with the socket server. In my case I created a generic function to take the name of the “room” from the XPage URL. The following client side JavaScript sends the “joinRoom” request to the server.
nickname=$('.username').text()
// Send message to server that user has joined WinX.nsf/xRoom1.xsp?open&login
var xpageName = location.href.split('.nsf/')[1] //xRoom1.xsp?open&login
xpageName = xpageName.split('.xsp')[0] //xRoom1
socket.emit('joinRoom', xpageName, nickname);
On the server we can then see the console.log as someone joins the room:
We can then have multiple users in individual “rooms” which are in this case 3 separate xPages xRoom1.xsp, xRoom2.xsp, xRoom3.xsp From the original chat room I am able to sen all three rooms a blanket message because they are all “listening” for “msg” as in the previous articles
socket.on('msg', function(data) {
console.log('msg here')
console.log(data)
redisClient.lpush('messages', JSON.stringify(data));
redisClient.ltrim('messages', 0, 99);
socket.broadcast.emit('msg', data); //broadcast to all users indiscriminately
});
Sending a message to a specific room
Modifying the original POST code which was shown in the previous article I was then able to create an XPage which will send a targetted message to a specific room. To do this, all I had to do was pass in an additional POST parameter of the room I wanted to send a message to.
I create a new Master XPage which had 3 fields on it – one for each room. Each of the individual fields had a “room” attribute which allows me to pick up a value to specify which room to send it to.
<div id="msgRoom1" class="msgCenter">
<input placeholder="Send a message to Room1" room="xRoom1" autocomplete="off" autofocus="autofocus" />
<button type="button">Send</button>
</div>
<br/><br/>
<div id="msgRoom2" class="msgCenter">
<input placeholder="Send a message to Room2" room="xRoom2" autocomplete="off" />
<button type="button">Send</button>
</div>
<br/><br/>
<div id="msgRoom3" class="msgCenter">
<input placeholder="Send a message to Room3" room="xRoom3" autocomplete="off" />
<button type="button">Send</button>
</div>
In the following code we bind to each of the buttons so that when the are clicked they:
set msgField to be the jQuery object representing the pertinent field
create the data object to send to the socket server
create the newMessage passing in data
The new message function then:
POSTs the data object at the “/roomPost” path on the server
var socketServerURL = (location.href.indexOf('copper.xomino.com')>0) ? "http://copper.xomino.com:3000" : 'http://xominosocket.mybluemix.net'
var socket = io.connect(socketServerURL)
$('.msgCenter Button').on('click', function(){
var msgField = $(this).prev('INPUT')
var data = { text: msgField.val(), nickname: nickname, when: new Date(), room: msgField.attr("room") };
sendMessageToRoom(data)
newMessage(data);
// Clear the message field
msgField.val('');
})
var sendMessageToRoom = function(data){
console.log(data)
$.ajax({ url:
socketServerURL+"/roomPost",
type: "POST",
data: data
}).done(function( msg ) {
console.log( msg);
});
}
We can see the message come through on the server
You can also see from the log that I was able to join 3 rooms from the Message Center page – this is as simple as creating three requests to join:
Site in action
The best way to see all this is to see it in action – as you can see from the video below.
Moving the application to Bluemix
Working locally I used to following code to determine if I was on copper.xomino.com or another server (demo.xomino.com or different again)
var socketServerURL = (location.href.indexOf('copper.xomino.com')>0) ? "http://copper.xomino.com:3000" : 'http://xominosocket.mybluemix.net'
var socket = io.connect(socketServerURL)
I had to make a slight change to my cors code in the local node server setup. I looked at the cors npm site to see how to dynamically add cors support and found the answer and modified my code accordingly. So you need to make sure you
//Marky adding CORS from copper
//app.use(cors()); //not used for whitelists
var whitelist = ['http://copper.xomino.com', 'http://demo.xomino.com', 'http://marky.psclistens.com'];
var corsOptions = {
origin: function(origin, callback){
var originIsWhitelisted = whitelist.indexOf(origin) !== -1;
callback(null, originIsWhitelisted);
}
};
marky.psclistens.com is the same local server as copper.xomino.com (I use a hosts file to make them both look at localhost). But this allows me to play with cors and in this case talk to the local or bluemix node server without needing another “Domino” server.
Because the site running locally is already connected to the Jazz Hub Git repository, as with the previous examples all I have to do is commit the changes locally, push to the Jazz Hub repository, the application will then be re-built and re-deployed to Bluemix.
It’s really about as simple as that. Because the code is already primed to check to see if we are looking locally or at Bluemix, a new URL for the application now looks at Bluemix.
So why do we need Bluemix again?
While the development was performed locally, the intention is to be able to create a capability which does not require the user to have a locally running node.js server. That is where Bluemix comes in. In essence what we are creating is a cloud hosted Websockets service which can be integrated into existing applications.
Conclusion
In this article we have seen that with a little creativity we are able to register different chat rooms within the same application. There are other ways to do this of course but the point of this example was to demonstrate the ability to send a specific message to specific users looking at an individual XPage.
In the next article we will take a look at the practical implications of this within an XPages application.
In this article I demonstrate how to create a Websocket message to be broadcast, from a website which is not connected to the Websocket directly. This will be achieved by using a http POST request of the message to a node.js server hosted on Bluemix and having that server then emit a Websocket message out to the application.
Introduction
In the previous article I demonstrated how to create an XPages chat client using a Bluemix hosted node.js server. I was able to do this by porting the node.js example to an XPage (xSocket.xsp). What I wanted to figure out (thank you Stack Overflow) was how to trigger a Websocket message to be sent out without using the Websocket socket.io JavaScript client.
Reference articles
This article is based on information already published on the following blog posts:
In the Watson translation service example article I discussed how to create a new POST route within your node application using the simple express construct “app.post”.
var cors = require('cors');
app.use(cors());
var corsOptions = {
origin: 'http://copper.xomino.com'
};
app.post('/xpages', cors(corsOptions), function(req, res){
.....
});
In a similar fashion in this example I am going to POST data from the copper.xomino.com website and therefore I need to add the cors code to the application to allow this.
var msgField = $('#msg'),
data = { msg: msgField.val(), nickname: nickname, when: new Date() };
So our target is to recreate that same data Object and send it through to the server socket emitter in the same format.
Creating the POST data
Once again learning from the previous Watson example the code for this is relatively simple. We create the data Object with a nickname and text value and then POST them to the xominosocket.mybluemix.net site. Logging the response from the server
var data = {
nickname: 'marky ajax',
text: 'Sample POST from AJAX'
}
console.log(data)
$.ajax({ url:
"http://xominosocket.mybluemix.net/xpages",
type: "POST",
data: data
}).done(function( msg ) {
console.log( msg);
});
Modifying the node.js code
The node.js code from the previous article needs to be updated in a number of ways.
The Watson Translation example used a version of express 3.x. Version 4 is more streamlined and does not contain a lot of middleware. When we attempt to POST data to the node server it cannot be understood because it is encoded by the web browser (by default). For his we need to add the npm module body-parser (not forgetting to include it in the package.json).
The app.post also needs to be able to parse the incoming POST and then re-purpose this into a socket emit event. We do this as follows. I also added a check to make sure the req.body was coming through. This was failing before I realized I needed body-parser.
// Handle the form POST containing the text and nickname
app.post('/xpages', cors(corsOptions), function(req, res){
var request_data = {};
console.log(req.body)
if (req.body){
request_data = {
'txt': req.body.text,
'nickname': req.body.nickname
};
} else {
request_data = {
'txt': 'this failed',
'nickname': 'Not this guy'
};
}
var data = { msg: request_data.txt, nickname: request_data.nickname, when: new Date() };
io.sockets.emit("msg", data);
res.send(data);
});
Testing the new capability
For the purposes of this article I am only going to show the example in Firebug console. As you can see from the code on the right we are posting to the xominosocket site. On the left you can see the communication was successfully sent to the site, the response and the headers:
From the next screenshot you can see that the message is them broadcast out to the other client currently connected to the socket, in this case in Chrome.
Cors security
The cors security only allows POSTing from the copper.xomino.com site. This is a good thing 🙂
Conclusion
In this article we have seen how we can “create” a websockets emission event on the server by POSTing a message to a separate page within the application.
While this may seem pointless (“because why couldn’t I just create a normal websockets message from the page”), what we will see in a future article is the need to emit an event from the server which is not connected to the Websocket and doesn’t need to be.
In this article I will demonstrate how I was able to take an example Bluemix, node.js based, websocket chat program and re-purpose it to be used in XPages.
Introduction
Earlier this year I was very excited to find the Websockets in XPages project on OpenNTF published by Mark Ambler. The concept behind that project is to be able to create a notes document in a queue which is processed and then send out to all users. As much as I promised to help out and use the project, life and a business need to learn and use Angular.js got in the way. My abject apologies to Mark for not following through on my promise to help move the project along.
With this article though, I want to start my exploration of using an external Websockets server to transmit messages to my XPages applications. One of the *nice* things about Websockets is that unlike JavaScript AJAX they are *not* restricted by CORS. This means that I can host my Websockets server anywhere. In this case it is a win win for me as I get to learn more about Bluemix, node.js, websockets and other NoSQL databases like Redis in this case.
It is not an especially difficult example to follow but I did find that it helped to understand a little about Bluemix and how a node.js application is put together. You should be able to figure it out though just following through the example.
Within the original example, the interface makes a connection to a Redis database to store the last 99 entries of the chat. Within the XPages example I could do that but I am not going to at this time. So the XPages interface will lose chat history when you refresh the page. I am not really concerned about that in the big picture.
Because I had all the files locally I was able to create a new IBM Domino database and drag the files into the WebContent directory. Within node.js the “Public” directory is assigned to the root of the server, but in this case I removed the public folder as it is unnecessary.
The original example uses the jade templating engine to create the web page. But in this case I felt lazy and just viewed the source of the example once it was working and then extracted all the HTML I needed.
Moving socket.io to the HEAD
Because this is XPages and because we have dojo and I am sure I have pointed out before – we have to move the files to the HTML HEAD in such a manner as they come before Dojo within the application. Socket.io is apparently one such JavaScript library.
The HTML of the XPage is relatively simple. As you can see below we a using the xp:resources tag to insert the same HTML references to the local files as they were in the original example.
Within the original example the socket code was on the same server as the client creating the messages. In this case they are not as my XPages server is wholly independent of Bluemix.
So I had to change the initial connection to the websocket server. Within the client.js file I changed the first line from
$(document).ready(function() {
var socket = io.connect('http://xominosocket.mybluemix.net') //connect to the Bluemix server
var nickname, msgList = $('#messages');
...
With these changes and a couple of stupid spelling mistake corrections I was able to bring my application up within my xSocket XPage.
You can see from the Firebug console that the copper.xomino.com application is talking to the xominosocket.bluemix.net application
Mobile Compatible
Yes you have to be connected to the website (rather than OS Push notification) but websockets works on iOS7+ and Android 4.4+
I realize I am not the first person to play with a websocket server on top of a domino database. I know of at least 4 other people who have at least done their own POC for this. But this is the first time I have got my hands on it 🙂 I’m a little excited……
Brief overview of WebSockets
I already wrote about the reasons for websockets in XPages. But to recap, WebSockets allows me the developer to PUSH messages out to logged in users without them asking for it. This reduces network traffic and creates a new paradigm for fluid data update.
What I will be doing over the next few blog posts is demonstrate examples of why WebSockets fundamentally changes how we architect our web apps in the future. Yes it is that significant in my mind.
Make sure you sign the update site before deploying
Install the update site in Domino Designer and restart
Understand that this is not port 80….this is demonstration right now and not production ready.
Once you have the plugin installed and up and running application you will be able to use the chat.nsf example to demonstrate for yourself what is going on. This is only intended to be an example of what is possible, not a production app. replacing sametime 🙂
From a Domino perspective, The core of the application is the websocket.nsf database. In there if you look at the Broadcast Example Agent you will see how it works.
This article was written based on code release 1.0.7
By creating a document in the database with the appropriate fields, the websocket OSGI code picks up the document from the view and routes it out to the users.
The websocket.nsf database tracks users in the vUsers view and from there we can ascertain the unique code assigned to each logged in user.
Using that websocketID listed in the view we are able to ensure that the appropriate message is transmitted to the appropriate user.
To extend this into something I can use I create an SSJS function which will look up the socketID – in this case I do not need to know the ID – just the username.
function createMessage(socketID, msgObj){
var db:NotesDatabase = session.getDatabase("","websocket.nsf",false);
var doc:NotesDocument = db.createDocument()
var online = ""
if (!socketID){
var view = db.getView("vUsers")
println("rec: "+msgObj.recipient)
var user:NotesViewEntry = view.getEntryByKey(msgObj.recipient)
if (user){
socketID = user.getColumnValues()[1]
online = user.getColumnValues()[2]
println(socketID)
}
}
if (socketID && (online=="ONLINE")){
doc.replaceItemValue("Form", "fmSocketMessage")
doc.replaceItemValue("from", msgObj.from)
doc.replaceItemValue("to", socketID)
doc.replaceItemValue("sentFlag", 0)
//The msgObj will come in as an Array of Json - we need to convert that into a multi-value notes field
//This is planned to be fixed to apss the JSON straight through in a later release
for (var i=0; i<msgObj.data.length; i++){
var temp = msgObj.data[i]
for (key in temp){
if (i<1){
var item:NotesItem = doc.replaceItemValue("data", key+"="+temp[key])
} else {
item.appendToTextList(key+"="+temp[key])
}
}
}
doc.replaceItemValue("text", "")
doc.save()
} else {
println("WebSocket messgae not sent")
}
}
I can then call this function from an XPages button (or anywhere else for that matter) and pass in the data as an object.
Looking at the chat example below which I have also slightly modified to console.dir the output so that it is readable we can see the message is sent to the client
In a more advanced demonstration I added a field to the right hand form so that i can pass through my own JSON – the video below shows this best 🙂
This was recorded at 1920 resolution so it appears blurred on youtube – CHANGE THE QUALITY to 720HD and that will fix it
Please bare in mind this is not production ready code – and it may never be……
Because there are two servers running (domino and websocket) the websockets are current running over a non-standard port. I believe IHS can be used as a proxy to fix that.
As a proof of concept though and to get the information out to the community and IBM (hint hint) this is a great step forward in capabilities.
This is a great example of using and OSGI plugin and the Java code in Domino to access a Java server and transmit data – very cool !
Conclusion
There is no polling of the server by the client. There are zero unnecessary http calls
We can have any application with an appropriate update, review the users who are logged in and update their web page with the appropriate data.
THIS IS AWESOME!!!!!!!!!!!!
Awesome!!!!!
Next – more examples
I feel a series coming on…….:)
Update
You know a project is moving along when this blog post was already out of date by the time I am ready to post it – but rather than waste it because there is good information in here – here’s the disclaimer.
This post is based on v1.0.7 of the webshell code release – as of v1.0.8 the JSON data is being passed via rich text which allows for pre-formatting server-side before sending it out. This also allows for more complex JSON creation than multi-value text fields can handle – thanks Mark – more about that soon !!
This change was based on my request for a feature on the github site
If you want to see this improve and progress – get involve – play with the code and make requests for changes – like everything OpenNTF – the more you play with it and the more people are involved – the better it will become !