EXTJS in XPages #9 – Infinite scrolling rebooted and reborn – v4.2 BufferedRenderer

In this article I will introduce the new EXTJS v4.2 Infinite scroller – the BuffererRenderer. The whole concept of infinite scrolling has been rewritten in the new version of the grid and it had made a huge difference to responsiveness and stability of the infinite grid.

EXTJS in XPages series

Here are links to all of the previous articles in this series

Demonstration

The EXTJS in XPages demonstration database can be found at http://demo.xomino.com/xomino/Extjs.nsf/xBufferedRenderer.xsp

Download

The sample database that will accompany this series is  available from the menu in the live demo database from this link – http://demo.xomino.com/xomino/Extjs.nsf

Introduction

The first thing I suggest you do is go and watch this blog post and video – it explains in WAY more detail that I possibly could how cool the new grid is.

Blog: First Look at Ext JS 4.2 Grid

Video: http://vimeo.com/59611158

And for those of you who couldn’t be bothered the upshot of all of this is that the grids are WAY better than ion 4.2. The synopsis from my perspective is:

  • 10,000 records are loaded into memory and only a fraction of them are displayed in the grid. As soon as a new row is scrolled into view and added to the grid’s DOM model – a previous row is removed. This makes for a small DOM and takes up a finite amount of browser memory.
    • The increase in XPiNC responsiveness is night and day – it is that much better
  • We no longer need remote filtering and sorting
    • thank goodness cos that was a pain and not as flexible as I wanted it to be anyway
  • We can easily take advantage of grouping a large number of records efficiently
  • Selecting multiple documents across the infinite scroller is retained (HUGE DEAL)
  • Grouping is seamless across the infinite scrolling
  • The only downside is that you have to wait for 10,000 records to load from the REST service – and that can be optimized as I have previous discussed
    • This also means we need to be smarter in reloading the grid only when we really need to
    • If it is a 100k download and takes 10 seconds to render – the user will wait the first time but not every time they need to open a document and then come back to the grid because it was not the one they wanted.
  • In some ways Buffered Render is not really an infinite scroller – because it does not go back to the server ever time the user needs to scroll down – it is really just a highly efficient document display modeler I guess.

So how do we make it work

Well there are some changes to our code but not too many and all of these should be familiar concepts based on the rest of the posts in this series so far.

First  we need to load the data into memory – we do this by calling a REST service. We are going to use a different REST service as well. In the previous infinite scroll example we needed the REST service to tell us how many documents there were in the remote store – we do not need that any more as we are going  to load every document in the view. We are going to use a flat JSON ViewService to pull all the documents from the ByFirstName view

The new REST Service

<xe:restService id="byFirstNameFlat" pathInfo="byFirstNameFlat">
	<xe:this.service>
		<xe:viewJsonService defaultColumns="true" count="10000&" viewName="byFirstName">
		</xe:viewJsonService>
	</xe:this.service>
</xe:restService>

Starting the process
The data is loaded in a different way for the bufferedRenderer. Instead of being loaded from a remote service it is loaded from a local data model. That is not a problem for us but means an extra step in the data management process.

The process kicks off in the following manner

Ext.onReady(function(){

	//We are going to pass to the getDataGrid function the url we want
	//and the callback function which we then wants executing once the data is retrieved
	//doing it this way gives us maximum control over what happens to the data
	//and makes the function more generic should we want to use it for other data
	//and other grids in the future

	var url = 'xRestService.xsp/byFirstNameFlat?count=100000',
			theData
	var theCallBack = function(theData){
			createGrid(theData)
			}
	getGridData(url, theCallBack)
});

In this function we “getGridData” and then passing in a callback function we determine what happens after the data is loaded. We are passing in the function variable theCallBack. This allows the getGridData fucntion to execute on what was passed in to it.

I cannot streaa enough how much of a good JavaScript coding practice this is. Instead of hard coding the getGridData function to call the next step in the process – we tell it what to do.
This creates a generic getGridData function which does nothing but that. It can then be used to do anything else we want – because we are able to tell it what to do next.

Check out my article on Make your XPages more maintainable – JavaScript Callback functions for a better understanding on how callback functions work.

function aRandomNumber() {
	return Math.floor(Math.random()*1000001)
}

function getGridData(url, callback){

	var dataURL = url
	$.ajax({
		  url: dataURL+&quot;&amp;rand=&quot;+aRandomNumber()+&quot;&amp;&quot;
		}).done(function ( data ) {
			//one the data is loaded then pass it to the callback function
			//passed in as one of the function arguments
			if(callback){
				callback(data)
			}
		});
}

Loading the data

$.ajax({
	url: 'xRestService.xsp/byFirstNameFlat'
	})
	.done(function ( data ) {
		createGrid2(data)
	});

The grid code has an additional feature for bufferedrenderer. This can either be added as a text string property in the grid or as a feature object of type bufferrenderer. If we do it this second way we have more control over its properties and it separates it from the main grid code.

buffRend = new Ext.grid.plugin.BufferedRenderer( {
        pluginId: 'buff1'
})

The grid
The grid has the renderer as a grid property.

&lt;strong&gt;The store&lt;/strong&gt;
	store = new Ext.data.Store({
		model : 'Person',
		autoLoad: true,
		sorters: {
			        property : 'firstName',
			        direction: 'ASC'
			    },
		id: 'store',
		data: data,
		remoteFilter: false,
		buffered: false,
        proxy: {
            type: 'memory'
        }
	});

The Grid
The grid itself is not that different from the previous examples except for the one critical line – plugins: buffRend

The BufferedRendered is added as a plugin by first defining the Ext.grid.plugin.BufferedRenderer and then adding it as a property to the grid during creation.

 var filters = {
        ftype: 'filters',
        // encode and local configuration options defined previously for easier reuse
        //encode: encode, // json encode the filter query
        local: local   // defaults to false (remote filtering)

	};

buffRend = new Ext.grid.plugin.BufferedRenderer( {
     pluginId: 'buff1'
})

	gridDock = [{
		id: 'activeStorePaging',
	     xtype: 'toolbar',
	     dock: 'bottom',
		 items: [
		   {
		       text: 'Clear Filter Data',
		       handler: function () {
		           grid.filters.clearFilters();
		       }
		   }
	    ]
	 }]

    var grid = Ext.create('Ext.grid.Panel', {
        renderTo: 'gridHere',
        frame: true,
        height: 400,
        title: 'Users',
	plugins: buffRend,
        features: [filters],
        store: store,
        iconCls: 'icon-user',
        columns: [{
            header: 'First',
            sortable: true,
            dataIndex: 'firstname',
            filterable: true
        }, {
            text: 'Last',
            sortable: true,
            dataIndex: 'lastname',
            field: {
                xtype: 'textfield'
            },
            filterable: true
        }, {
            text: 'Address',
            sortable: true,
            dataIndex: 'address',
            field: {
                xtype: 'textfield'
            },
            filterable: true
        }, {
            text: 'City',
            sortable: true,
            dataIndex: 'city',
            field: {
                xtype: 'textfield'
            },
            filterable: true
        },{
            text: 'State',
            width: 80,
            sortable: true,
            dataIndex: 'state',
            field: {
                xtype: 'textfield'
            },
            filterable: true
        }],
        // paging bar on the bottom
        bbar: gridDock
    });

}

Performance
If you watched the video you will see how the improvements in performance are handled. In the example shown in the video they show that for a few thousand records there are hundreds of thousands of DOM elements created for an old flat grid to be scrolled through. Using the new bufferedRenderer there are always the same number loaded – more in the order of 2,000 regardless of the overall data size. This is acheived by programmatically adding an removing the table rows to the visible grid as the user scrolls up and down the grid. If the user can see 25 rows then only 27 rows are loaded. The grid Tbale displaying the data is constantly modified as the user scrolls up and down.

The difference between 4.1.1 and 4.2 is night and day. I don’t just say that lightly. In the latest application we were working on, with 4.1.1 we were seeing a slowdown in XPiNC around 200 documents in the view. With 4.2 we have loaded over 3000 documents without any significant reduction in responsivness and/or speed of the grid.

Night and Day – it is a different application to interact with – quite remarkable!!

Conclusion
Looking at this demo alone on a modern browser you would not really notice much of a difference – but if you compare some of the other examples in XPiNC to this one you will really notice the improvement. That larger the data set the better then improvement.

EXTJS in XPages #8 – Selecting data from the grid and opening a document

In this article I will show you how to get data upon selecting a document – I will then demonstrate how we open a document from the grid.

EXTJS in XPages series

Here are links to all of the previous articles in this series

Demonstration

The EXTJS in XPages demonstration database can be found at http://demo.xomino.com/xomino/Extjs.nsf/xGridOpenDocument.xsp

Download

The sample database that will accompany this series is  available from the menu in the live demo database from this link – http://demo.xomino.com/xomino/Extjs.nsf

Selection Model

The selection model for a grid is very easily retrieved using the dblclick event within the grid:

 var selection =  grid.getSelectionModel().getSelection();
 var fieldValue = selection[0].data.fieldname

Listeners

To detect the double click event we have to add a listener to the grid. There are many listeners for the grid and more information on them can be found here in the Sencha help files

grid = Ext.create('Ext.grid.Panel', {
	listeners: {      //listener config setting for the grid
		events HERE.....................
	}
}

Getting the UNID

In out view we are going to add a  new column docid – very simple @Text(@DocumentUniqueID) column formula. But this column is not going to be displayed in the view – it is going to be hidden.

We add the new column and add the “hidden: true” property to it in the grid so that it is not visible to the user. But this does mean that it is available to us programmatically.

{
	text: 'docid',
	width: 10,
	dataIndex: 'docid',
	hidden: true
},

(Note: the @UNID property is passed in by the REST service and *yes* we could use that instead of adding a new column – however I feel that this is easier to explain and does not require a discussion regarding the usage of a property of the REST service JSON array )

Opening the document

In our example grid we are simply going to open our document by getting the docid field from the hidden column and opening a window using a manually generated URL string

So all together or grid code now looks like this:

grid = Ext.create('Ext.grid.Panel', {
	renderTo: 'gridHere',
	frame: true,
	features: [filters],
	height: 400,
	title: 'Users',
	store: store,
	iconCls: 'icon-user',
	listeners: {      //listener config setting for the grid
		itemdblclick: function(dataview, record, item, index, e) {
			console.log('dblClick Event')
			var selection = grid.getSelectionModel().getSelection()
			window.open('xPerson.xsp?action=readDocument&documentId='+selection[0].data.docid)
		}
	}
}

As you can see from the example page this is simple, effective and completely poor as a user experience – but this article was about getting data from the grid – not about making a cool application 🙂

Conclusion

Programmatically accessing data in the grid is a very powerful technique when attempting to manipulate the EXTJS grids and insert them into your application.

EXTJS in XPages #7 – Doing an @Unique(@DbColumn) equivalent in the grid

In this article I will show you how to make a simple @Unique(@DbColumn)) equivalent in the EXTJS grid

Introduction

When you have a notes view/EXTJS grid you often want to be able to do a lookup of all the unique values in a column – this is especially useful when you are also applying filters in the grid.

What is particularly useful is that the values in the column are uniqued automatically – this allows you to reduce down the values which are presented to the user to select from. This also means not having to create some elaborate search/filter capability going back to the server to change the lookup.

Caveat

This only works on the data in the current view so will not work if you are using remote filtering. In a later article I will demonstrate the new 4.2 BufferedRenderer which makes front end filtering on 10,000 documents a reality

The code

The code is very simple and can be demonstrated easily by adding a ComboBox to the grid – in this example the combobox is appended to the grid and the values come from the state field in the grid store

new Ext.form.ComboBox({
        renderTo: 'gridHere',
        store: grid.getStore().collect('state')
    });

ext1

Filtering the grid reduces the data

ext2

and then running the add combobox again now creates one with a reduced set – NOT the uniquing of the values automatically

ext3

To see the returned value we just use firebug and execute the collect method – you can see that the values are returned as an array – this could then be parsed into any field.

ext4

Conclusion

Using the EXTJS grid can be very much akin to using a notes view – the trick is having the knowledge to know what is possible.

PS

The data is returned in the order that it is presented in the grid – if you want alphabetically then so an array.sort()

🙂

EXTJS in XPages #6 – remote sorting using the REST service

In this article I will demonstrate how use the remote sorting feature from within the EXTJS grid

EXTJS in XPages series

Here are links to all of the previous articles in this series

Demonstration

The EXTJS in XPages demonstration database can be found at http://demo.xomino.com/xomino/Extjs.nsf/xGridWithInfiniteScroller.xsp

Download

The sample database that will accompany this series is  available from the menu in the live demo database from this link – http://demo.xomino.com/xomino/Extjs.nsf

Introduction

As we saw in the previous article we are able to utilize the infinite scroller to provide users a long list of documents to scroll through without taking up excessive memory at any one time. The local downside of this is that when you click on a column header to sort the column it only sorts the documents in view and not the whole list.

Using the sort property of the REST service we are able to perform this task simply and effectively.

The grid properties

Within the grid we have to signify that we are using remote sorting. We do this using the remoteSort property of the store:

remoteSort: true,

This tell the grid to use the remote sort capability when the user click on a column header.

The parameters sent to the REST service

When the user clicks on a column header a URL call is made to the REST service to reload the data. Two additional parameters are sent to the service through the Query_String “sort=” and “dir”

Parameters sent to the REST service

_dc 1363479015660
count 20
dir ASC
keys
limit 20
page 4
sort address
start 60

The sort parameter signifies teh field to be sorted and the dir (ASC or DESC) tells which direction

The View

Within the notes view, to make this capability possible, we must have the view column sortable in both directions.

ext1

This does have one potentially enormous downside……view index size. Magnified by the number of columns which all need to be sortable in both directions and especially if you have Readers and Authors fields in the documents. But I am sure you know about these risks and I will not bore you with them – just remember it is a consideration when using this capability.

The REST service

The REST service has two properties we will need: sortColumn and sortOrder. Looking closely at the sortOrder though we see that the REST service is expecting “ascending” or “descending” as its options. We are going to have to turn ASC and DESC into ascending and descending appropriately.

Within the REST service properties we can see the sortColumn and sortOrder options. Selecting to compute the sortOrder we add the following SSJS code to the property:

<xe:this.sortOrder>
<![CDATA[#{javascript:
	var direction = context.getUrlParameter('dir');
		switch(direction)
		{
		case 'ASC':
			return 'ascending'
			break;
		case 'DESC':
			return 'descending'
			break;
		default:
			return 'ascending'							}
		}]]>
</xe:this.sortOrder>

This code takes the incoming “dir” UrlParameter and returns the expected ascending or descending value based on the JavaScript switch/case statement.

In the sortColumn property we do a similar thing by taking the incoming UrlParameter, but in the case no changes are needed because the field name already matches the programmatic column name in the view

context.getUrlParameter('sort');

Conclusion
And that is it – we have seen in this article how we can use SSJS in a REST service property to change the incoming UrlPramter to return the value needed by the REST service and in doing that we have enabled remote sorting of view columns.

Searching XPages REST service and returning the real count in the results.

In this article I will show you how I managed to get the search results count into a REST service and why I wanted to do that.

Introduction

Back in August 2012 I posted this on Stackoverflow. I will answer the question there but I don’t feel like I have “answered” it, just worked around it.

How do I make the XPages REST control return the overall number of filtered matches?

http://stackoverflow.com/questions/11786930/how-do-i-make-the-xpages-rest-control-return-the-overall-number-of-filtered-matc

Problem

When you create a viewItemFileService REST service using the the ExtensionLibrary control you get a response which looks like this:

{
    "@timestamp":"2013-03-12T23:25:57Z",
    "@toplevelentries":1299,
    "items":
    [
      {
          "@entryid":"100-8DE5EBF5E35C17F285257B1B00254A8F",
          "@unid":"8DE5EBF5E35C17F285257B1B00254A8F",
          "firstname":"Benjamin",
          "lastname":"Imran",
          "address":"2112 Hall Street",
          "city":"Las Vegas",
          "state":"NV",
          "zip":89121,
          "country":"US",
          "email":"Benjamin.A.Wayman@mailinator.com"
      },

which is fine…and from that you can ascertain that there are 1299 documents in the view (@toplevelentries)

The problem is when you execute a search against this REST service (for example ‘[address] contains street’) you still get @toplevelentries: 1299

This causes me great angst when it comes to the EXTJS grids because I want to use the Infinite Scrolling capability. What that does is calculates based on the size of the view (1299) how large a scroll bar to create and then display 25 entries at a time and load them as necessary. This is really smart except after you have searched the REST service. The grid still thinks there are 1299 documents even when there is only a subset matching the search results. This causes a lot of blank scrolling 😦

In my example the search ‘[address] contains street’ returns 338 matches. But this causes the following to happen with the infinite grid

rest1

And that sucks. Also note 52 pages in the pager!!!!

The EXTJS code to get the number of documents in the service looks like this:

 var store = Ext.create('Ext.data.Store', {
        // allow the grid to interact with the paging scroller by buffering
        buffered: true,
        pageSize: 25,
        leadingBufferZone: 25,
        autoLoad: true,
        autoDestroy: true,
        autoSync: true,
        model: 'Person',
        proxy: {
            type: 'rest',
            url: 'xRestService3.xsp/marky?count=25',
            reader: {
                type: 'json',
                root: 'items',
                totalProperty : '@toplevelentries' // This is the determining value
            },
            writer: {
                type: 'json'
            }
        }
    });

And as you can see form the code @toplevelentries is always 1299

So how do we get the 338 number to the grid without causing:

  • Some major hacktastic, back end to front end, extra ajax calls, and  redrawing the grid on the fly shenanigans
  • Manually re-writting the REST service from scratch (which I could do but would cause much pain and suffering)
  • Performance issues with XPiNC connecting to the server
  • My eyes to bleed…..

This has baffled me for months and here’s how I got around it.

Solution

In the REST service you can create a custom column. In this case we are just returning the requestScope value for markyCount in the column “theRealCount”:

<xe:this.columns>
	<xe:restViewColumn name="theRealCount">
		<xe:this.value>
		<![CDATA[#{javascript: return requestScope.get("markyCount")}]]>
		</xe:this.value>
	</xe:restViewColumn>
</xe:this.columns>

And markyCount is going to be the 338 value from the search. So how do we get that?

The REST service has an id value which can be computed – and I really don’t care what the id is because I am not binding it to a control on the form. So I used the opportunity for some server side computation of the id to create a little FTSearch.

In the example below I am simply getting the view used by the REST service (byFirstName) doing an FTsearch on it and then putting the count in the markyCount requestScope.

<xe:this.id>
	<![CDATA[${javascript:
		var theView:NotesView = database.getView("byFirstName")
		var count = theView.FTSearch("[address] contains street");
		requestScope.put("markyCount", count)
		return "searchMe" //irrelevant value
		}
	]]>
</xe:this.id>

So what this does is create a REST service output which looks like this:

{
    "@timestamp":"2013-03-12T23:25:57Z",
    "@toplevelentries":1299,
    "items":
    [
      {
          "@entryid":"100-8DE5EBF5E35C17F285257B1B00254A8F",
          "@unid":"8DE5EBF5E35C17F285257B1B00254A8F",
          "firstname":"Benjamin",
          "lastname":"Imran",
          "address":"2112 Hall Street",
          "city":"Las Vegas",
          "state":"NV",
          "zip":89121,
          "country":"US",
          "email":"Benjamin.A.Wayman@mailinator.com",
          "theRealCount": 338  //this is the new column
      },

Note theRealCount column with 338 in it. This is then added to every column output by the REST service. So yes the downside is that we are adding computation and output data size to the feed but it appears to make an insignificant amount of difference to the output speed of the service.

We then change the EXTJS code to look at the theRealCount value in the first Item and not @toplevelentries

 var store = Ext.create('Ext.data.Store', {
        // allow the grid to interact with the paging scroller by buffering
        buffered: true,
        pageSize: 25,
        leadingBufferZone: 25,
        autoLoad: true,
        autoDestroy: true,
        autoSync: true,
        model: 'Person',
        proxy: {
            type: 'rest',
            url: 'xRestService3.xsp/marky?count=25',
            reader: {
                type: 'json',
                root: 'items',
                totalProperty : 'items[0].theRealCount' // This is the determining value
            },
            writer: {
                type: 'json'
            }
        }
    });

And we get this – no more blank lines and 14 pages in the pager 🙂

rest2

EXTJS in XPages #5 – Infinite scroller

In this article I will show how simple it is to turn your paging grid into an infinite scrolling one.

EXTJS in XPages series

Here are links to all of the previous articles in this series

Demonstration

The EXTJS in XPages demonstration database can be found at http://demo.xomino.com/xomino/Extjs.nsf/xGridWithInfiniteScroller.xsp

Download

The sample database that will accompany this series is  available from the menu in the live demo database from this link – http://demo.xomino.com/xomino/Extjs.nsf

Introduction
In the last article I show how to add a pager to your grid to prevent a significant number of documents from loading into your grid at once. This is more analogous to the view panel out of the box paging that we are familiar with in the XPages environment.

From a user interaction perspective paging is good and allows the user to sort through each 25 documents worth of data. But it fails miserably when you have to page more then 4 times to get more data. That task of having to page down through the data becomes very cumbersome very quickly.

Infinite scrolling

The concept of infinite scrolling was first made popular on websites like Facebook and Twitter. When the user gets to the bottom of the screen an icon appears indicating more data loading and then the user continues to scroll. This has now become the expectation of users trying to sift through a lot of data on the web.

Add in infinite scrolling to your grid

Adding infinite scrolling is as simple as 3 lines of additional code in your existing grid. This feature is only supported as of EXTJS 4.1 only.

1. We require the paging scroller library to be loaded
2. We add a line to the grid signifying that this is a buffered data set
3. We Did another parameter to allow a “scroll ahead” before the next data set is loaded.

Looking at the video below you can see that as the user starts to scroll the page seems to just go and go – and only when dragging the scroll bar down quickly do you even see a Loading Mask. Adding Firebug console you can see that the buffered pages are loaded into memory before the user gets to the point of the grid where they would need to be displayed.

Here is the code sections which have changed since the last article.


Ext.require([
       	'Ext.data.*',
       	'Ext.grid.*',
        'Ext.toolbar.Paging',
        'Ext.grid.PagingScroller'       //THIS LINE
]);

var store = Ext.create('Ext.data.Store', {
    // allow the grid to interact with the paging scroller by buffering
    pageSize: 25,
    buffered: true,               //THIS LINE
    leadingBufferZone: 25,        //THIS LINE
    autoLoad: true,
    autoDestroy: true,
    autoSync: true,
    model: 'Person',
    proxy: {
        type: 'rest',
            url: 'xRestService.xsp/byFirstName?count=25',
            reader: {
                type: 'json',
                root: 'items',
                totalProperty : '@toplevelentries'
            },
            writer: {
                type: 'json'
            }
        }
    });

Performance

This is a huge performance boost from loading all the documents into the grid at once. When using the grid purely as a reporting display model, you should consider using the infinite scrolling capabilities of the grid as a standard. There are some caveats though to be aware of; when it comes to interactivity with the grid data this can cause some challenges (see below)

This performance boost is most significant in IE and XPiNC where a few hundred documents can slow down the browser. With infinite scrolling you can have 10,000 documents in a view and scroll to any of them.

Better than Paging?

Absolutely:

  • A familiar interface for users experienced in using the notes client
  • A much faster way to find the user to find the page containing their document of reference.
  • Less bandwidth wasted guessing. Moving the scroll bar up and down to a visual guide is more accurate than guessing the number of the page where the document might be based on the sorting of the column in question.

What about filtering?

Infinite scrolling does have a couple of downsides if you want to use other parts of the grid functionality. Because you are only showing a few of the documents at a time – filtering the “loaded documents” is not going to give you anything and you have to go with remote filtering. Remote filtering is quite possible in the XPage environment but I am unable to share at this time as it was part of a client engagement where we implemented this.

What about sorting?

Also with sorting you can sort the local documents in the grid – but that doesn’t really help. You have to use remote sorting which can be done and we will look at in the next article.

Custom Control

Adding infinite scrolling to the custom control is easy because the only things which have changed are the 3 lines of Javascript. This has been updated in the demo database.

Conclusion

Paging is better than loading all documents into the grid at once. Infinite scrolling is better than Paging in most cases. Like everything in this series, there are pros and cons which have to be considered for each piece of functionality and it is important to understand them so that you can verbalize these considerations to your customer when architecting a solution.

EXTJS in XPages #4 – Adding a Pager

In this article I will demonstrate how to add a pager control to your basic grid.

Demonstration

The EXTJS in XPages demonstration database can be found at http://demo.xomino.com/xomino/Extjs.nsf/xGridWithPager.xsp

Download

The sample database that will accompany this series is  available from the menu in the live demo database from this link – http://demo.xomino.com/xomino/Extjs.nsf

Introduction

As we saw from the previous articles we can make a basic EXTJ grid from a REST service. Loading a huge number of documents into an EXTJS grid does have its drawbacks though:

  • Poor user experience to have to search down through hundreds of records
  • Browser memory management issues.

Too many documents? *Warning*

Unlike traditional notes which allows for a personal cache of data for the user through the notes client, the web browser is not capable of doing this. Any browser (some worse than others) is going to run into problems when you load a significant amount of data on a page. In my experience the worst browser for this is the XPiNC browser (Firefox 3.5 code base), then IE then Firefox and Chrome.

As the developer you also have to consider:

  • load time
    • how long will it take to load hundres of documents from the REST service over your network
  • parsing time for the grid library
    • The EXTJS code has to take the JSON and create from it a gigantic table and add that to the DOM. This takes time and browser resources – too slow and the user gets impatient.

Recommendation

If you have more than 250 documents do not load them all into the grid at any one time. For all of the reasons above, starting with the user experience. While Notes Client or Excel may be able to allow a user to manage thousands of records, a web browser is not – find a better way.

Adding a Paging toolbar

There are only a couple of steps to adding a paging toolbar to the bottom of your grid.

First we need to add a property to the store – so that the grid knows how many documents are coming in

   var store = Ext.create('Ext.data.Store', {
        autoLoad: true,
        autoDestroy: true,
        autoSync: true,
        model: 'Person',
        proxy: {
            type: 'rest',
            url: 'xRestService.xsp/byFirstName?count=25',    // ?count=25 sets the number of docs the REST sends each time
            reader: {
                type: 'json',
                root: 'items',
                totalProperty : '@toplevelentries'     //This comes from the REST service
            },
            writer: {
                type: 'json'
            }
        }
    });

The REST service provides the @TopLevelEntries – this is the number of documents in the view


{
    "@timestamp":"2013-03-02T16:09:08Z",
    "@toplevelentries":1299,  // <----------
    "items":
    [
      {
          "@entryid":"1-66F3A5FE2B2A38CC85257B1B00254E6D",
          "@position":"1",
          "firstname":"Adam",
          "lastname":"Saenz",
          "address":"2519 Custer Street",
          "city":"ROCKWOOD (SOMERSET)",
          etc etc,

We then need to add the code to the EXT.require to include the Paging toolbar

Ext.require([
             	'Ext.data.*',
             	'Ext.grid.*',
                'Ext.toolbar.Paging',
         ]);

and finally add the toolbar to the grid itself by adding it as a property after the columns

        // paging bar on the bottom
        bbar: Ext.create('Ext.PagingToolbar', {
            store: store,
            displayInfo: true,
            displayMsg: 'Displaying topics {0} - {1} of {2}',
            emptyMsg: "No topics to display"
        })

And that’s it.

ext41

The 1 of 52 is calculated by the grid 1299 documents divided by 25 documents at a time = 52 pages

Custom Control update

We can add the pager to the XPages Custom Control in the same way as previously described in the last article – but in the case we need to add another property – how many documents per page instead of how many documents for the overall feed.

        proxy: {
		            type: 'rest',
		            url: '#{javascript:facesContext.getExternalContext().getRequest().getRequestURI()+"/ccREST?count="+compositeData.countNum}',
		            reader: {
		                type: 'json',
		                root: 'items',
		                totalProperty : '@toplevelentries'
		            },
<xe:restService pathInfo="ccREST" id="restServe1" >
	<xe:this.service>
		<xe:viewItemFileService  defaultColumns="true" viewName="${javascript:compositeData.theViewName}"></xe:viewItemFileService>
	</xe:this.service>
</xe:restService>

ext42

In this case you can see we are using the whole ByFirstName view and the generic Custom Control displayed all the columns successfully !

This Custom Control is available in the download database (ccEXTJSGridwithPager)

EXTJS in XPages #3 – Creating a basic grid from a Custom Control

In this article I will demonstrate how to create an EXTJS grid representation of a view using a custom control and 3 custom properties. At the end of this article you will be able to drag and drop a custom control onto your XPage, give it the count, the viewName and the name of the placement element and a grid will be created for you without any further coding.

Introduction

In XPages we want to be able to genericize everything down to a re-usable custom control and in this example I will show you how to add a grid to your XPage doing just that. The following XPage source code is all that is needed to create this basic grid capability.

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" xmlns:xc="http://www.ibm.com/xsp/custom">

	<xc:ccBasicExtJSGrid countNum="10" putGridHere="gridHere" theViewName="ByName-First5Col" ></xc:ccBasicExtJSGrid>

	<div id="gridHere" class="extjsGrid"></div>

</xp:view>

Demonstration

The EXTJS in XPages demonstration database can be found at http://demo.xomino.com/xomino/extjs.nsf

Download

The sample database that will accompany this series is  available from the menu in the live demo database from this link – http://demo.xomino.com/xomino/Extjs.nsf

The Custom Control

The Custom Control is created with 3 Properties which are populated when it is added to the XPage. You add it to your XPage by dragging and dropping and adding the 3 properties:

xt31

The View

The view ByName-First5Col is a simple view of 5 columns containing a single field in each column but as we will see this control will handle any number of columns.

ext32

The code

The end result is to create the JavaScript necessary to create a Basic grid in the same fashion as I did in the first article. But in this case we are not providing the column names or headers, we are programmatically getting them from the view itself.

The fields are created by going to the view and getting the columns and then of those, the item name (nvc.getItemName())

var fields=[
		#{javascript:
				var view:NotesView = database.getView(compositeData.theViewName)	//get the view
				var msg=[];
				for (var i=1; i<view.getColumnCount()+1; i++){
					var nvc:NotesViewColumn = view.getColumn(i);			//get each view column in turn
					msg.push('\''+nvc.getItemName()+'\'')				//get the column programmatic name
				}
				return msg.join(',')							//return the string of fields
		}
	]

The columns are created in the same maner but instead of getting just the column Item Name we get the column Title (nvc.getTitle() as well)

var columns=[
	#{javascript:
			var view:NotesView = database.getView(compositeData.theViewName)
			var msg=[];
			for (var i=1; i<view.getColumnCount()+1; i++){
				var nvc:NotesViewColumn = view.getColumn(i);
				//Create the JSON object containing the column programmatic name and the column title form the view
				msg.push('{header: \''+nvc.getTitle()+'\', sortable: true, dataIndex: \''+nvc.getItemName()+'\'}')
			}
			return msg.join(',')
		}
	]

I added the REST control to the Custom Control itself for simplicity and to make it more portable. This does have the restriction (at this time) of only having one grid on a page at a time. The pathInfo and id of a REST control cannot apparently be computed at build time so I have to keep them static. I am sure there is a way around it…in the mean time the viewName is passed in via compositeData as well as the count. As I think about it you could probably give control to the user using scoped variables and have them select the number of documents displayed…but that is for later.

	<xe:restService pathInfo="ccREST" id="restServe1" >
		<xe:this.service>
			<xe:viewJsonService defaultColumns="true" count="${javascript:compositeData.countNum}" viewName="${javascript:compositeData.theViewName}"></xe:viewJsonService>
		</xe:this.service>
	</xe:restService>

The viewName is also passed into the EXTJS code so that it knows where to look for the REST service on the page.

    var store = Ext.create('Ext.data.Store', {
        autoLoad: true,
        autoDestroy: true,
        autoSync: true,
        model: 'Person',
        proxy: {
            type: 'rest',
            url: '#{javascript:facesContext.getExternalContext().getRequest().getRequestURI()+"/ccREST"}',
            reader: {
                type: 'json',
                root: 'items'
            },
            writer: {
                type: 'json'
            }
        }
    });

The EXTJS code is then simplified because we are passing in the fields and columns values already computed above. Below is the code for the whole Custom Control.

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" xmlns:xe="http://www.ibm.com/xsp/coreex">
	<script type="text/javascript" src="ext-4.1.1/ext-all.js"></script>
	<link rel="stylesheet" type="text/css" href="ext-4.1.1/resources/css/ext-all.css" />

	<style>
		.icon-user { background-image: url(images/user.png) !important; }
		.icon-add { background-image: url(images/add.png) !important; }
		.icon-delete { background-image: url(images/delete.png) !important; }
	</style>

	<xp:scriptBlock id="scriptBlock1">
		<xp:this.value><![CDATA[

		var fields=[
			#{javascript:
					var view:NotesView = database.getView(compositeData.theViewName)
					var msg=[];
					for (var i=1; i<view.getColumnCount()+1; i++){
						var nvc:NotesViewColumn = view.getColumn(i);
						msg.push('\''+nvc.getItemName()+'\'')
					}
					return msg.join(',')
			}
		]

		columns=[
			#{javascript:
					var view:NotesView = database.getView(compositeData.theViewName)
					var msg=[];
					for (var i=1; i<view.getColumnCount()+1; i++){
						var nvc:NotesViewColumn = view.getColumn(i);
						msg.push('{header: \''+nvc.getTitle()+'\', sortable: true, dataIndex: \''+nvc.getItemName()+'\'}')
					}
					return msg.join(',')
				}
			]

		Ext.Loader.setConfig({
		  enabled : true,
		  disableCaching : false
		});

		Ext.require([
		             	'Ext.data.*',
		             	'Ext.grid.*',
		             	'Ext.ux.grid.FiltersFeature'
		         ]);

		Ext.define('Person', {
		    extend: 'Ext.data.Model',
		     fields: fields
		});

		Ext.onReady(function(){

		    var store = Ext.create('Ext.data.Store', {
		        autoLoad: true,
		        autoDestroy: true,
		        autoSync: true,
		        model: 'Person',
		        proxy: {
		            type: 'rest',
		            url: '#{javascript:facesContext.getExternalContext().getRequest().getRequestURI()+"/ccREST"}',
		            reader: {
		                type: 'json',
		                root: 'items'
		            },
		            writer: {
		                type: 'json'
		            }
		        }
		    });

		    var local = true;

		     var filters = {
		        ftype: 'filters',
		        // encode and local configuration options defined previously for easier reuse
		        //encode: encode, // json encode the filter query
		        local: local   // defaults to false (remote filtering)

			};

		    var grid = Ext.create('Ext.grid.Panel', {
		        renderTo: 'gridHere',
		        frame: true,
		        features: [filters],
		        height: 400,
		        title: 'Users',
		        store: store,
		        iconCls: 'icon-user',
		        columns: columns
		    });
		});
			]]></xp:this.value>
	</xp:scriptBlock>

	<xe:restService pathInfo="ccREST" id="restServe1" >
		<xe:this.service>
			<xe:viewJsonService defaultColumns="true" count="${javascript:compositeData.countNum}" viewName="${javascript:compositeData.theViewName}"></xe:viewJsonService>
		</xe:this.service>
	</xe:restService>
</xp:view>

Results

As you can see below – a grid, same as we had before.

extjs33

But now we can add a new column to view and without changing the control itself the new column appears on the web.


ext34

ext35

And even complex columns with combined field values.

ext37 ext36

Conclusions

With this Custom control we are able to begin to create the ability to add these EXTJS grids to our XPages, very quickly and very effectively. As we go through the series I will update the custom control with the new capabilities as we look at new grids. Eventually we will have a plethora of controls which can be used to create powerful grids in next to no time at all.

In the mean time you can download the control from the sample database and play with it yourself. http://demo.xomino.com/xomino/Extjs.nsf

EXTJS in XPages #2 – Basic Grid capabilities

In this article I will demonstrate the basic out of the box functionality which comes with each grid and will focus on the capabilities of each column.

Introduction

In the previous article I demonstrated how to add a basic grid to your XPage using the EXTJS grid control and the ExtLib REST service. What we will look at in this article is the basic capabilities of the view columns.

Re-sizing columns

If you mouse over a column separator at the top you will see a familiar icon – clicking on the separator you can drag and drop the column increasing and decreasing the size

ext11 ext12 ext13

Re-ordering of columns

If you click on a column and “drag it” over to the left and right you can re-order them.

ext21

ext22

ext23

Hiding columns

If you mouse over the top right of a column you will see a drop down icon appear – click on it and you will be able to hide one of the columns. You can actually programmatically hide the columns when the grids open and have the users make them visible later if you want to.

ext25

ext26

Sorting Columns

If you click on the column header – or click on the small drop down and select to order the column you can do that

ext24

Filtering Columns

Well ok this is not a BASIC feature of the grid as it is added as another required library – but I figured I would show it cos it is COOL and should be a basic premise of the grids you create. Code will be provided in a later article but for now just know that it can be done locally (filtering the grid contents) and remotely (asking the server for a filtered results set) and it is in the basic demo site


ext6ext7ext8

Demonstration

Go look at the basic demo site and see for yourself.

 

EXTJS in XPages #1 – Starting from scratch (the first grid)

In this article I will demonstrate how to create a basic EXTJS grid on your XPages website. I will go through each step of the process and once complete this will be the reference for all following articles on how to do it.

Introduction

EXTJS is a JavaScript library. Well ok to be clear it is MANY JavaScript libraries which can all be added to your XPages in the same way.  I will show you where to find the files, how to add them to your Domino database and then how to use the Extension Library REST service to put your first grid on your XPage.

Sencha

Originally built as an add-on library extension of YUI by Jack Slocum, Ext JS includes interoperability with jQuery and Prototype. Beginning with version 1.1, Ext JS retains no dependencies on external libraries, instead making their use optional.

Licensing

Sencha has two licenses commercial and GPL. For the sake of this blog I am working under the GPL license because everything is being made public. If you want to use the grid for your work you will need to pay for it. A developer pack is $600 each and at a commercial XPages developers charge out rate that is less than a day….I think it will make you more productive!

Downloading

Go to the download page and download the appropriate copy of the whole library (45M-ish).

Getting it into your Database

Once downloaded, unzip the file and get the contents on your machine. That is 210M of unreal goodness and excessive volume and we REALLY don’t need to bloat our database with all that. So delete all the folders except resource and src and remove all the js files until you are left with the extjs files and the license….

ext9

That has got us down to about 30M of files which is still a LOT but they are not going to be loaded into your webpage all at the same time so don’t fret about it

Then drag and drop the root folder into your databases WebContents folder. This can be found in the Package Explorer via  the XPage perspective in 8.5.3 or Window/Show Eclipse Views/Java in <8.5.3.
ext10

Asynchronous Module Definition

As you may know with the dojo libraries there is a methodology called “Asynchronous Module Definition (AMD)” which allows a JavaScript application to load in the modules that it needs – and this rocks! What is also means is that (and I know some of you were thinking it) you are not going to load “30M” worth of files onto your webpage at any time.

Building an XPage

On your XPage you will need to reference the EXTJS library to make sure that is included in your webpage resources. This is done by simply adding the following code to the XPage:

<script type="text/javascript" src="ext-4.1.1/ext-all-debug.js"></script>	
<link href="ext-4.1.1/resources/css/ext-all.css" rel="stylesheet" type="text/css" /> 

This also adds the basic stylesheet to the grid as well (more on that in a later blog post) Once that is added to the XPage then we are all ready to start……

Looking at the examples

As I mentioned in my first article – you really should go look at the examples page in this case specifically the grids section. In there you should poke around and look at the code, terrifying as it may be it really shouldn’t be and as we go through this blog series I will explain and simply all the examples. In this case I am going to look at the RESTful Store example  Clicking on the restful.js link you will see all the code used int his example. I am not going to go into the code itself in this example, suffice as to say you just need to copy and paste 🙂

Creating a grid from a REST service
We are going to start our XPage with Dave Leedy’s fakenames first name view ext1 from that we are going to look at the view in Designer, specifically at the programmatic names of the columns. ext4 As you can see from the image above, each of the columns has a programmatic use name. This is important to remember and necessary to change sometimes.

The REST service.

The REST service is added to an XPage in the following manner – (you must have the Extension Library installed on the client and the server to make this happen). This is the code on my xRestService XPage for the ByFirstName feed. Very simple and sends 100 documents out in JSON format

<xe:restService id="by-FirstName" pathInfo="byFirstName">
	<xe:this.service>
	<xe:viewItemFileService viewName="ByName-First"
		count="100" defaultColumns="true" systemColumns="132">
	</xe:viewItemFileService>
	</xe:this.service>
</xe:restService>

Looking at the REST service through a URL call we see the following format for the output:

{
    "@timestamp":"2013-02-23T06:53:34Z",
    "@toplevelentries":1300,
    "items":
    [
      {
          "@entryid":"1-66F3A5FE2B2A38CC85257B1B00254E6D",
          "@unid":"66F3A5FE2B2A38CC85257B1B00254E6D",
          "@noteid":"688E",
          "@position":"1",
          "@siblings":1298,
          "firstname":"Adam",
          "lastname":"Saenz",
          "address":"2519 Custer Street",
          "city":"ROCKWOOD (SOMERSET)",
          "state":"PA",
          "zip":15557,
          and so on,
      },
      {
          "@entryid":"2-3857882D6726A78F85257B1B00254F1C",
          "@unid":"3857882D6726A78F85257B1B00254F1C",
          "@noteid":"6B4A",
          "@position":"2",
          and so on,

Adding the Grid to the XPage I have simplified the example given on the Sencha site so that we can just see the bare bones of the grid. As we go through th series I will add more and more complication into the grids so that we can see how they work. The grid code is built up of multiple components:

  • Require the necessary EXTJS libraries
  • Defining a People Object
  • Creating the data store
  • Getting the grid data from the REST service
  • Applying the data to the grid in columns

So let’s look at the code The AMD part

Ext.require(['Ext.data.*', 'Ext.grid.*']);

The Person Object In here we define the fields ‘firstname’, ‘lastname’, ‘address’, ‘city’, ‘state’. These field names relate to the names of the fields coming in through the REST service JSON string and therefore directly to the programmatic column name in the view (told you it was important)

Ext.define('Person', {
    extend: 'Ext.data.Model',
    fields: [{
         'firstname', 'lastname', 'address', 'city', 'state']
});

The Data Store In this section the data Store is created which is used to populate the grid. The store: defines the Person Object (above) as the model to display the information sets up the connection to get the REST service from the ‘xRestService.xsp/byFirstName’ URL

    var store = Ext.create('Ext.data.Store', {
        autoLoad: true,
        autoSync: true,
        model: 'Person',
        proxy: {
            type: 'rest',
            url: 'xRestService.xsp/byFirstName',
            reader: {
                type: 'json',
                root: 'items'
            },
            writer: {
                type: 'json'
            }
        }
    });

The Grid itself Finally we build the grid – and the grid parameters define what it looks like – I have added comments to the code so that you can see in place what each section is doing

    var grid = Ext.create('Ext.grid.Panel', {
        renderTo: 'gridHere',		//render to the id named gridHere
        width: 600,			//width of the grid
        height: 300,			//height of the grid
        frame: true,
        title: 'Users',			//Grid title at the top
        store: store,			//Store object
        iconCls: 'icon-user',		//icon for the title
        columns: [{			//for each column in the grid define the attributes of the column
            header: 'First',		//Column Header Title
            width: 80,			//Column width
            sortable: true,		//Is the column sortable?
            dataIndex: 'firstname',	//what is the data field in the JSON used to fill this column
            field: {			//what type of field is this?
                xtype: 'textfield'
            }
        }, {
            text: 'Last',
            width: 80,
            sortable: true,
            dataIndex: 'lastname',
            field: {
                xtype: 'textfield'
            }
        }, {
            text: 'Address',
            width: 80,
            sortable: true,
            dataIndex: 'address',
            field: {
                xtype: 'textfield'
            }
        }, {
            text: 'City',
            width: 80,
            sortable: true,
            dataIndex: 'city',
            field: {
                xtype: 'textfield'
            }
        },{
            text: 'State',
            width: 80,
            sortable: true,
            dataIndex: 'state',
            field: {
                xtype: 'textfield'
            }
        }]
    });
});

And there it is – you add all the above in a SCRIPT tag on the XPage the code for you first grid. We also add a little CSS to show the icon for the image and a div as a container. This is not the “best” way to do it, and we will discuss that in a later article but for the moment it is the easiest to understand being in the same XPage.

<script type="text/javascript" src="ext-4.1.1/ext-all-debug.js"></script>
	<link href="ext-4.1.1/resources/css/ext-all.css" rel="stylesheet" type="text/css" /></pre>
<style><!--
.icon-user {
    background-image: url(images/user.png) !important;
}

.icon-add {
    background-image: url(images/add.png) !important;
}

.icon-delete {
    background-image: url(images/delete.png) !important;
}
--></style>
<pre>
<script type="text/javascript">// <![CDATA[
Ext.require(['Ext.data.*', 'Ext.grid.*']);

Ext.define('Person', {
    extend: 'Ext.data.Model',
    fields: [{
        name: 'id',
        type: 'int',
        useNull: true
    },  'firstname', 'lastname', 'address', 'city', 'state']
});

Ext.onReady(function(){

    var store = Ext.create('Ext.data.Store', {
        autoLoad: true,
        autoSync: true,
        model: 'Person',
        proxy: {
            type: 'rest',
            url: 'xRestService.xsp/byFirstName',
            reader: {
                type: 'json',
                root: 'items'
            },
            writer: {
                type: 'json'
            }
        }
    });

    var grid = Ext.create('Ext.grid.Panel', {
        renderTo: 'gridHere',
        width: 600,
        height: 300,
        frame: true,
        title: 'Users',
        store: store,
        iconCls: 'icon-user',
        columns: [{
            header: 'First',
            width: 80,
            sortable: true,
            dataIndex: 'firstname',
            field: {
                xtype: 'textfield'
            }
        }, {
            text: 'Last',
            width: 80,
            sortable: true,
            dataIndex: 'lastname',
            field: {
                xtype: 'textfield'
            }
        }, {
            text: 'Address',
            width: 80,
            sortable: true,
            dataIndex: 'address',
            field: {
                xtype: 'textfield'
            }
        }, {
            text: 'City',
            width: 80,
            sortable: true,
            dataIndex: 'city',
            field: {
                xtype: 'textfield'
            }
        },{
            text: 'State',
            width: 80,
            sortable: true,
            dataIndex: 'state',
            field: {
                xtype: 'textfield'
            }
        }]
    });
});
// ]]></script></pre>
<div id="gridHere"></div>
<pre>

ext5

Demonstration

The EXTJS in XPages demonstration database can be found at http://demo.xomino.com/xomino/extjs.nsf

You will note that the menu does not look right – I know – one thing to discuss with the grids – their stylesheets…..we will see 🙂

This was tested in IE9, Safari, Chrome, Firefox, Chrome on Android and iPhone/iPad

Play with the columns and see what they can do – I will post again soon to discuss the capabilities of the basic grid further.

Download

The sample database that will accompany this series will always be available from the menu in the live demo database from this link – http://demo.xomino.com/xomino/Extjs.nsf

Conclusion

I think it is safe to say that this is a simple, fantastic, aesthetically pleasing, way of showing data on the screen to the users and hopefully this was a simple enough process to follow along with. This is a simple grid with not much functionality but as we will come to see – there are w whole world of different capabilities within the EXT grids and their plugins 🙂