How to tell if your web application has been modified in 4 lines of code

In this article I will demonstrate a simple but effective way to determine if something has changed on your webpage. I will use the MD5 hashing library available in dojo and compares the hash at the start of the process and at the end – if it has changed – something has changed.

Acknowledgements
I have to acknowledge Alex Skinner and Jason Lavoie who first showed me this method many years ago on a .NET project and Troy Reimer who was the √úber smart genius to take the idea and apply it so eloquently to XPages using dojo cryptographic libraries. Thanks guys ūüôā

Introduction

It has been an age old issue with web form development – how do I know if the user has changed anything?

We could track the start and end value of every field – doable but laborious. Also comes to a stumbling block with you have user interactions which are not field based – like dragging and dropping an employee from one manager box to another.

We could use the XSP._isDirty() function which is provided for us in XPages but that does not apply in all situations and this example will work with any webpage (assuming dojo is loaded)

MD5 Hash

Partially because of my lack of understanding but entirely because of its lack of relevance to this solution I am not going to go into how MD5 hashing works but to suffice to say: Рan MD5 hash is an iterative encryption algorithm which can reduce a string of any length down to a fixed length (in our case 16 characters).

Dojo Cryptography libraries

Dojo provides various cryptographic algorithms and you can find out more about them here

http://dojotoolkit.org/reference-guide/1.6/dojox/encoding/crypto.html

In our case we are going to use the MD5 library and we add it to our XPage in the following manner:

	dojo.require("dojox.encoding.digests.MD5");

What needs to be checked?

Well it just so happens (if you hadn’t noticed) that HTML is basically a long text string. So what we are going to do is to put a panel around the form fields we are trying to test again.

<xp:panel styleClass="myPanel">
	web form stuff here
</xp:panel>

And then once the page is loaded we are going to create the MD5 Hash of the HTML within that panel like this.

var oDocHash = oDocHash= dojox.encoding.digests.MD5(dojo.query('.myPanel')[0].innerHTML);
console.log(oDocHash)

We can then either stick the hash in a cookie – or in a field on the form – or in this case in a windows variable.
(Before the JavaScript purists start to have a go – I know, I know don’t use the windows object to store variables – tough – I said 4 lines of code, didn’t I) ūüėČ

So then the overall code to create the hash looks like this:

dojo.require("dojox.encoding.digests.MD5");
window.docHash = dojox.encoding.digests.MD5(dojo.query('.myPanel')[0].innerHTML);

The checking

When you are ready – or the user is not ready to check if anything has changed all we have to do is re-hash and then compare the variables, like this

hasChanged = (window.docHash != dojox.encoding.digests.MD5(dojo.query('.myPanel')[0].innerHTML))? "This form has changed" : ""
if (hasChanged){alert(hasChanged)}

The overall Code sample

//Load the necessary dojo Crypto Library
dojo.require("dojox.encoding.digests.MD5");

//Get a hash of the innerHTML of the element with the .myPanel class
//Run this when the page is loaded
window.docHash = dojox.encoding.digests.MD5(dojo.query('.myPanel')[0].innerHTML);
console.log("Stored hash: "+window.docHash)

//determine if a change has been made
//run this when the page is ready to be saved or navigated away from
hasChanged = (window.docHash != dojox.encoding.digests.MD5(dojo.query('.myPanel')[0].innerHTML))? "This form has changed" : ""
console.log("New hash: "+dojox.encoding.digests.MD5(dojo.query('.myPanel')[0].innerHTML))

//panic panic
if (hasChanged){alert(hasChanged)}

The demonstration

As you can see from the video below we are starting out with a simple XPage – not in Edit mode – I have also added some console logs to the code so that you can see what is going on. I use firebug to alter the HTML – and the hashes do not match. For the icing on the cake though if I put the HTML back to be exactly what it was before we started then the hashes match.

Conclusion

And there you have it – it is simple to implement and you don’t need to understand how the cryptography works to use it. Generally better than way I find because trying to understand cryptography usually causes headaches and nose bleeds.

Advertisements

EXTJS in XPages #6 – remote sorting using the REST service

In this article I will demonstrate how use the remote sorting feature from within the EXTJS grid

EXTJS in XPages series

Here are links to all of the previous articles in this series

Demonstration

The EXTJS in XPages demonstration database can be found at http://demo.xomino.com/xomino/Extjs.nsf/xGridWithInfiniteScroller.xsp

Download

The sample database that will accompany this series is ¬†available from the menu in the live demo database from this link –¬†http://demo.xomino.com/xomino/Extjs.nsf

Introduction

As we saw in the previous article we are able to utilize the infinite scroller to provide users a long list of documents to scroll through without taking up excessive memory at any one time. The local downside of this is that when you click on a column header to sort the column it only sorts the documents in view and not the whole list.

Using the sort property of the REST service we are able to perform this task simply and effectively.

The grid properties

Within the grid we have to signify that we are using remote sorting. We do this using the remoteSort property of the store:

remoteSort: true,

This tell the grid to use the remote sort capability when the user click on a column header.

The parameters sent to the REST service

When the user clicks on a column header a URL call is made to the REST service to reload the data. Two additional parameters are sent to the service through the Query_String “sort=” and “dir”

Parameters sent to the REST service

_dc 1363479015660
count 20
dir ASC
keys
limit 20
page 4
sort address
start 60

The sort parameter signifies teh field to be sorted and the dir (ASC or DESC) tells which direction

The View

Within the notes view, to make this capability possible, we must have the view column sortable in both directions.

ext1

This does have one potentially enormous downside……view index size. Magnified by the number of¬†columns which all need to be sortable in both directions and especially if you have Readers and Authors fields in the documents. But I am sure you know about these risks and I will not bore you with them – just remember it is a consideration when using this capability.

The REST service

The REST service has two properties we will need: sortColumn and sortOrder. Looking closely at the sortOrder though we see that the REST service is expecting “ascending” or “descending” as its options. We are going to have to turn ASC and DESC into ascending and descending appropriately.

Within the REST service properties we can see the sortColumn and sortOrder options. Selecting to compute the sortOrder we add the following SSJS code to the property:

<xe:this.sortOrder>
<![CDATA[#{javascript:
	var direction = context.getUrlParameter('dir');
		switch(direction)
		{
		case 'ASC':
			return 'ascending'
			break;
		case 'DESC':
			return 'descending'
			break;
		default:
			return 'ascending'							}
		}]]>
</xe:this.sortOrder>

This code takes the incoming “dir” UrlParameter and returns the expected ascending or descending value based on the JavaScript switch/case statement.

In the sortColumn property we do a similar thing by taking the incoming UrlParameter, but in the case no changes are needed because the field name already matches the programmatic column name in the view

context.getUrlParameter('sort');

Conclusion
And that is it – we have seen in this article how we can use SSJS in a REST service property to change the incoming UrlPramter to return the value needed by the REST service and in doing that we have enabled remote sorting of view columns.

jQuery in XPages #18 Galleria (Easy to use Picture Carousel)

In this article I will describe how to implement and use the Galleria jQuery plugin to create a powerful picture carousel within your XPages application

Demonstration

The XPages integration of Galleria is demonstrated here

Download

The demonstration database can be downloaded from the link above or from here

Galleria

Introduction

“Galleria is a JavaScript image gallery framework that simplifies the process of creating beautiful image galleries for the web and mobile devices.”

I came across this carousel while I was researching my talk for IBMConnect 2013 and I was amazed by the simplicity, how powerful it was and how in depth and easy to understand the help documentation was. This article may be short and sweet because the plugin is that easy to implement!

Take a look at the example page http://galleria.io/

How does it work?

Looking at the documentation page¬†there is a myriad of information on how to make the carousel you really want. The Beginner’s Guide is one of the best I have seen for any plugin. It is a step by step¬†instruction¬†on how to create the carousel in your web page including how to add jQuery.

We have to change that slightly to make the carousel work in our XPage – but only a little….

Download the plugin files from here http://galleria.io/download/

We add the¬†jQuery library¬†and the Galleria¬†library¬†file(s) to the database by adding them to the Web-Contents folder accessible via the package explorer.¬†¬†(Window‚Äď>Show Eclipse Views‚Äď>Package Explorer) and drag and drop the two libraries from your computer into the ‚Äújs‚ÄĚ folder. We also add the themes and css files to the WebContent folder in the galleria folder.

gal1

We add the libraries to our XPages like this in the source panel

<script src="js/galleria-1.2.8.min.js"></script>

Using the information on the Beginner’s guide we also add the style

<style>
	#galleria{ width: 700px; height: 400px; background: #000 }
</style>

as well as the code to load the carousel once the page is loaded

Galleria.loadTheme('galleria/themes/classic/galleria.classic.min.js');
Galleria.configure({
   transition: 'fade',
   lightbox: true,
   imageCrop: true
});
Galleria.run('#galleria');

Once this is added to the page we just need to add some images…..In the beginners guide it staes that we have to just add some images into the galleria DIV

<div id="galleria">
    <img src="photo1.jpg">
    <img src="photo2.jpg">
    <img src="photo3.jpg">
</div>

but in our case we going to pull the pictures from notes documents in a view. In the same way as we did for the prettyphoto lightbox example we are going to add a view as a data source and use a repeat control to add the images to the page itself.

The data source

<xp:this.data>
	<xp:dominoDocument var="imageDoc" formName="imageHolder"></xp:dominoDocument>
	<xp:dominoView var="imagesView" viewName="images"></xp:dominoView>
</xp:this.data>

and the repeat control placing the images inside the galleria DIV

<div id="galleria">
	<xp:repeat id="repeat1" rows="30" value="#{imagesView}" var="imageRepeat">
		<xp:image id="Image1">
			<xp:this.url><![CDATA[#{javascript:"#{imageRepeat.imagePath}"}]]></xp:this.url>
		</xp:image>
	</xp:repeat>
</div>

And that is the absolute basic carousel (not that much different from the prettyphoto lightbox really at this point”

So what’s the difference with this one then?

Well it looks different, that is often a deciding factor when trying to impress your customer – which do they prefer the look of?

There are significantly more options to this carousel than the prettyphoto lightbox and this carousel has been optimized for mobile which the prettyPhoto has not (as far as I can tell). Let’s take a look at some of them…

Galleria comes with the following options:

  • Theming (allowing the developer to use pre-build settings to change the colors surrounding the carousel and the speed of the show)
  • Fullscreen capability
  • Responsive mobile view with swipe actions
  • An extensive API

In real words what does this mean? It means that you have almost absolute control over everything which happens within the slideshow or within the picture popups. Look at the optional themes as well. There are different themes you can buy which would serve you well if you were trying to implement this in a commercial site or if you just wanted to pump up an internal site.

In my example I have added a number of options to the carousel which give it some features. You should look seriously into the API to see what is possible.

Galleria.loadTheme('galleria/themes/classic/galleria.classic.min.js');
	Galleria.configure({
		transition: 'fade',
		lightbox: true,
		showCounter: true,
		showInfo: true,
		showImageNav: true,
		imagePan: true,
		imageCrop: true
	});
Galleria.run('#galleria');

Demonstration

Check out the demo(s)

The XPages integration of Galleria is demonstrated here

TakingNotesPodcast 176 Cloudability – What I learned about cloud services.

Yesterday I was asked to co-host the taking notes podcast ep176 with Bruce Elgort where we talked with Mat Ellis of cloudability.com.

I have to admit that I was not “up” on cloud services very much before Bruce asked me to stand in for Julian and co-host the show. In the past 3 days my eyes have been opened up to the possibility of Amazon AWS and other cloud services. For instance did you know that Amazon hosts free virtual machines for you to use?

Let’s say for example you have multiple customers running different versions of the notes client and/or Internet Explorer and you need different test environments – Rather than keep a development instance of designer and run it locally on your machine you can run it “in the cloud” and remote into the machine and use it there.

So what I hear you say? Well for a start it is always available, from anywhere, from any machine – (internet connectivity permitting) and that provides you some serious advantages:

  1. Connectivity from any machine anywhere
  2. no need to backup the VM on your computer
  3. no need to keep the disk space yourself on your laptop and/or SAN/WAN
  4. You only need it when you need it – turn it off when you are not using it
  5. Shared access for customers and other developers that you do not have on a local virtual machine.

And this is just from a personal perspective – just imagine what an IT department could use this for?

Ever need a load of servers quickly? And you thought you had to buy them? now you can test a clustered xpages app in the cloud with 50 servers if you wanted to – use them¬†for an hour –¬†and then puff

When you start to scale it up and down for IT departments to use there are obviously costs involved, and that is where cloudability helps to come in and track the usage numbers for you – making sure you are not overspending on your cloud investment.

If your interest if piqued listen to the podcast and see what you can learn about cloud cost management and the possibilities.

It is always worth broadening your mind to know what is possible. Even if it is not relevant to your daily work today, it could be tomorrow

Searching XPages REST service and returning the real count in the results.

In this article I will show you how I managed to get the search results count into a REST service and why I wanted to do that.

Introduction

Back in August 2012 I posted this on Stackoverflow. I will answer the question there but I don’t feel like I have “answered” it, just worked around it.

How do I make the XPages REST control return the overall number of filtered matches?

http://stackoverflow.com/questions/11786930/how-do-i-make-the-xpages-rest-control-return-the-overall-number-of-filtered-matc

Problem

When you create a viewItemFileService REST service using the the ExtensionLibrary control you get a response which looks like this:

{
    "@timestamp":"2013-03-12T23:25:57Z",
    "@toplevelentries":1299,
    "items":
    [
      {
          "@entryid":"100-8DE5EBF5E35C17F285257B1B00254A8F",
          "@unid":"8DE5EBF5E35C17F285257B1B00254A8F",
          "firstname":"Benjamin",
          "lastname":"Imran",
          "address":"2112 Hall Street",
          "city":"Las Vegas",
          "state":"NV",
          "zip":89121,
          "country":"US",
          "email":"Benjamin.A.Wayman@mailinator.com"
      },

which is fine…and from that you can ascertain that there are 1299 documents in the view (@toplevelentries)

The problem is when you execute a search against this REST service (for example ‘[address] contains street’) you still get @toplevelentries: 1299

This causes me great angst when it comes to the EXTJS grids because I want to use the Infinite Scrolling capability. What that does is calculates based on the size of the view (1299) how large a scroll bar to create and then display 25 entries at a time and load them as necessary. This is really smart except after you have searched the REST service. The grid still thinks there are 1299 documents even when there is only a subset matching the search results. This causes a lot of blank scrolling ūüė¶

In my example the search ‘[address] contains street’ returns 338 matches. But this causes the following to happen with the infinite grid

rest1

And that sucks. Also note 52 pages in the pager!!!!

The EXTJS code to get the number of documents in the service looks like this:

 var store = Ext.create('Ext.data.Store', {
        // allow the grid to interact with the paging scroller by buffering
        buffered: true,
        pageSize: 25,
        leadingBufferZone: 25,
        autoLoad: true,
        autoDestroy: true,
        autoSync: true,
        model: 'Person',
        proxy: {
            type: 'rest',
            url: 'xRestService3.xsp/marky?count=25',
            reader: {
                type: 'json',
                root: 'items',
                totalProperty : '@toplevelentries' // This is the determining value
            },
            writer: {
                type: 'json'
            }
        }
    });

And as you can see form the code @toplevelentries is always 1299

So how do we get the 338 number to the grid without causing:

  • Some major hacktastic, back end to front end, extra ajax calls, and ¬†redrawing the grid on the fly shenanigans
  • Manually re-writting the REST¬†service¬†from scratch (which I could do but would cause much pain and suffering)
  • Performance issues with XPiNC connecting to the server
  • My eyes to bleed…..

This has baffled me for months and here’s how I got around it.

Solution

In the REST service you can create a custom column. In this case we are just returning the requestScope value for markyCount in the column “theRealCount”:

<xe:this.columns>
	<xe:restViewColumn name="theRealCount">
		<xe:this.value>
		<![CDATA[#{javascript: return requestScope.get("markyCount")}]]>
		</xe:this.value>
	</xe:restViewColumn>
</xe:this.columns>

And markyCount is going to be the 338 value from the search. So how do we get that?

The REST service has an id value which can be computed – and I really don’t care what the id is because I am not binding it to a control on the form. So I used the opportunity for some server side computation of the id to create a little FTSearch.

In the example below I am simply getting the view used by the REST service (byFirstName) doing an FTsearch on it and then putting the count in the markyCount requestScope.

<xe:this.id>
	<![CDATA[${javascript:
		var theView:NotesView = database.getView("byFirstName")
		var count = theView.FTSearch("[address] contains street");
		requestScope.put("markyCount", count)
		return "searchMe" //irrelevant value
		}
	]]>
</xe:this.id>

So what this does is create a REST service output which looks like this:

{
    "@timestamp":"2013-03-12T23:25:57Z",
    "@toplevelentries":1299,
    "items":
    [
      {
          "@entryid":"100-8DE5EBF5E35C17F285257B1B00254A8F",
          "@unid":"8DE5EBF5E35C17F285257B1B00254A8F",
          "firstname":"Benjamin",
          "lastname":"Imran",
          "address":"2112 Hall Street",
          "city":"Las Vegas",
          "state":"NV",
          "zip":89121,
          "country":"US",
          "email":"Benjamin.A.Wayman@mailinator.com",
          "theRealCount": 338  //this is the new column
      },

Note theRealCount column with 338 in it. This is then added to every column output by the REST service. So yes the downside is that we are adding computation and output data size to the feed but it appears to make an insignificant amount of difference to the output speed of the service.

We then change the EXTJS code to look at the theRealCount value in the first Item and not @toplevelentries

 var store = Ext.create('Ext.data.Store', {
        // allow the grid to interact with the paging scroller by buffering
        buffered: true,
        pageSize: 25,
        leadingBufferZone: 25,
        autoLoad: true,
        autoDestroy: true,
        autoSync: true,
        model: 'Person',
        proxy: {
            type: 'rest',
            url: 'xRestService3.xsp/marky?count=25',
            reader: {
                type: 'json',
                root: 'items',
                totalProperty : 'items[0].theRealCount' // This is the determining value
            },
            writer: {
                type: 'json'
            }
        }
    });

And we get this – no more blank lines and 14 pages in the pager ūüôā

rest2

EXTJS in XPages #5 – Infinite scroller

In this article I will show how simple it is to turn your paging grid into an infinite scrolling one.

EXTJS in XPages series

Here are links to all of the previous articles in this series

Demonstration

The EXTJS in XPages demonstration database can be found at http://demo.xomino.com/xomino/Extjs.nsf/xGridWithInfiniteScroller.xsp

Download

The sample database that will accompany this series is ¬†available from the menu in the live demo database from this link –¬†http://demo.xomino.com/xomino/Extjs.nsf

Introduction
In the last article I show how to add a pager to your grid to prevent a significant number of documents from loading into your grid at once. This is more analogous to the view panel out of the box paging that we are familiar with in the XPages environment.

From a user interaction perspective paging is good and allows the user to sort through each 25 documents worth of data. But it fails miserably when you have to page more then 4 times to get more data. That task of having to page down through the data becomes very cumbersome very quickly.

Infinite scrolling

The concept of infinite scrolling was first made popular on websites like Facebook and Twitter. When the user gets to the bottom of the screen an icon appears indicating more data loading and then the user continues to scroll. This has now become the expectation of users trying to sift through a lot of data on the web.

Add in infinite scrolling to your grid

Adding infinite scrolling is as simple as 3 lines of additional code in your existing grid. This feature is only supported as of EXTJS 4.1 only.

1. We require the paging scroller library to be loaded
2. We add a line to the grid signifying that this is a buffered data set
3. We Did another parameter to allow a “scroll ahead” before the next data set is loaded.

Looking at the video below you can see that as the user starts to scroll the page seems to just go and go Рand only when dragging the scroll bar down quickly do you even see a Loading Mask. Adding Firebug console you can see that the buffered pages are loaded into memory before the user gets to the point of the grid where they would need to be displayed.

Here is the code sections which have changed since the last article.


Ext.require([
       	'Ext.data.*',
       	'Ext.grid.*',
        'Ext.toolbar.Paging',
        'Ext.grid.PagingScroller'       //THIS LINE
]);

var store = Ext.create('Ext.data.Store', {
    // allow the grid to interact with the paging scroller by buffering
    pageSize: 25,
    buffered: true,               //THIS LINE
    leadingBufferZone: 25,        //THIS LINE
    autoLoad: true,
    autoDestroy: true,
    autoSync: true,
    model: 'Person',
    proxy: {
        type: 'rest',
            url: 'xRestService.xsp/byFirstName?count=25',
            reader: {
                type: 'json',
                root: 'items',
                totalProperty : '@toplevelentries'
            },
            writer: {
                type: 'json'
            }
        }
    });

Performance

This is a huge performance boost from loading all the documents into the grid at once. When using the grid purely as a reporting display model, you should consider using the infinite scrolling capabilities of the grid as a standard. There are some caveats though to be aware of; when it comes to interactivity with the grid data this can cause some challenges (see below)

This performance boost is most significant in IE and XPiNC where a few hundred documents can slow down the browser. With infinite scrolling you can have 10,000 documents in a view and scroll to any of them.

Better than Paging?

Absolutely:

  • A familiar interface for users experienced in using the notes client
  • A much faster way to find the user to find the page containing their document of reference.
  • Less bandwidth wasted guessing. Moving the scroll bar up and down to a visual guide is more accurate than guessing the number of¬†the¬†page where the document might be based on the sorting of the column in question.

What about filtering?

Infinite scrolling does have a couple of downsides if you want to use other parts of the grid functionality. Because you are only showing a few of the documents at a time – filtering the “loaded documents” is not¬†going¬†to give you anything and you have to go with remote filtering. Remote filtering is quite possible in the XPage environment but I am unable to share at this time as it was part of a client engagement where we implemented this.

What about sorting?

Also with sorting you can sort the local documents in the grid – but that doesn’t really help. You have to use remote sorting which can be done and we will look at in the next article.

Custom Control

Adding infinite scrolling to the custom control is easy because the only things which have changed are the 3 lines of Javascript. This has been updated in the demo database.

Conclusion

Paging is better than loading all documents into the grid at once. Infinite scrolling is better than Paging in most cases. Like everything in this series, there are pros and cons which have to be considered for each piece of functionality and it is important to understand them so that you can verbalize these considerations to your customer when architecting a solution.

Agenda Posted for DCLUG

The agenda has been posted for the next DCLUG (see below). To kick off the year I will be presenting my jQuery session from this years IBM Connect conference.

Please take this opportunity if you are in the area and able to, attend and contribute to the IBM Domino community in the DC area and find out how “we” can help ourselves.

 

Agenda (20 Mar 2013)
————————————————————-

11:30am-12:00pm
Networking (lunch provided by IBM)
12:00pm-12:15pm
Welcome
Introductions
Who are we? Why are we here? What can we contribute?
Planned Meeting schedule
Need for meeting rooms
Need for speakers

12:15pm – 1:00pm

Presentation
Mark Roden: ¬†jQuery – the world’s most popular JavaScript library comes to XPages
1:00pm-1:30pm
Questions/Answers Networking
1:30pm
END

 

Location

IEG Briefing Center

600 14th Street, NW , Washington, DC

 

For more information and to sign up for the event – check out

http://www.meetup.com/DC-Lotus-Professionals/events/105313582/