Writing a single library for SSJS and CSJS validation – first success

The other day I wrote about my failure to realize that CSJS and SSJS libraries shalt not cross  during my attempt to write a single library for CSJS and SSJS validation code.

Not to be out done and with some great suggestions from the wonderful Sven Hasselbach I trucked on into the next attempt.

Parameterized development

I first met this concept a few years ago. I was working on an application created by John McCann, and all the script routines for a suite of applications were sorted in notes documents. The application would look up the code it needed and then executed it as needed. This made for extremely efficient code management and the ability to make updates to code snippets without changing any “code”.

Using this design pattern I set out to create a simple document in a view with a simple JavaScript function within it – the “are the fields the same” code I used in the previous article.

js1

I then created a Java Bean to access the notes view and get the field value.

(Yes I could have done it in SSJS – but hey this is a learning exercise as much as anything and the more Java code crosses my path (as rare as it is) the more likely I am to get familiar with it)

The bean is pretty simple and I am not going to detail the how’s and why’s and when’s of the bean – if you want to learn more about beans go see Russ Maher !!

package com.xomino.jsLookup;
//load import for Domino classes
import java.io.Serializable;

import lotus.domino.*;
import lotus.domino.local.Database;
import lotus.domino.local.Document;
//import for JavaServer Faces classes
import javax.faces.context.FacesContext;
//var marky = com.xomino.jsLookup.getValidationCode.getCode
public class getValidationCode implements Serializable{

	/**
	 *
	 */
	private static final long serialVersionUID = 1L;
	private String theCode;

	public getValidationCode() {
		System.out.println("Starting Bean");
	}

	// open method, this actually runs the whole App
	  public String getCode(){
	      System.out.println("MARKY SSJS");
	        // let's add a try catch here, to grab errors near the end
		    try {
		      // Declare Variables, one to hold Documents/ and page through results with the other
		      Document doc;
		      //BEGIN DEBUG
		      Database database = (Database) FacesContext.getCurrentInstance().getApplication()
		      				.getVariableResolver().resolveVariable(FacesContext.getCurrentInstance(), "database");
		      System.out.println("Database Obtained..." + database);
		      // Find the view in question
		      View view = database.getView("jsValidation");
		      System.out.println("View Obtained..." + view);
		        //get the document
		        doc = (Document) view.getFirstDocument();
		        System.out.println("Doc Obtained..." + doc);
		        System.out.println("Loading to Xpage...");
		        // process the document in the View
		        //recycle to free up mem
		        theCode = doc.getItemValueString("js");
		        System.out.println("Loading to Xpage...");
		        doc.recycle();
		    } catch (Exception e) {
		      e.printStackTrace();
		      System.out.println("FAIL");
		    }
			return theCode;
	  }

}

Big thanks to the Java Guru known as David Leedy for pointing out my inability to watch notesin9 videos correctly and learn how to do beans

Anyway the interesting code

The code which is stored in the notes document is just a simple JavaScript function

function checkPasswords(){
    if (getComponent("Password").getSubmittedValue() != getComponent("ConfirmPassword").getSubmittedValue()){
      return false;
    } else {
      return true;
    }
}

Back in my XPage I set up two sections for code – one for SSJS and one for CSJS

The CSJS code which will run on the web page looks like this (return lookupCode.getCode())

<xp:this.resources>
	<xp:script clientSide="true">
		<xp:this.contents>
			<![CDATA[${javascript:
				return lookupCode.getCode()
        		}]]>
		</xp:this.contents>
	</xp:script>
	<xp:script src="/libValidation.js" clientSide="true"></xp:script>
</xp:this.resources>

In this context the text string returned from the bean feeds directly into the contents of the script library – looks like this in the page source:

js3

The keen eyed among you will have noticed I also included the libValidation.js function which I created in the last article which is the CSJS equivalent of the getComponent(x).getSubmittedValue()

The SSJS code is similar but with one crucial difference:

<xp:validateExpression message="Passwords Must Match"><!-- (2) -->
	<xp:this.expression>
		<![CDATA[#{javascript:
			var theCode = lookupCode.getCode()
			eval(theCode)
			return checkPasswords()
		}]]>
	</xp:this.expression>
</xp:validateExpression>

In the SSJS I have to use “eval” to evaluate the text string which is returned from the bean. This works just fine – but eval is very very evil and I do not like this approach at all. But right now we are talking proof of concept so I am ok with it in development but would never use this in production. If the contents of the lookup document were compromised this would expose a massive hole in the security of the entire server – but that is a blog post for another day……

But the same code (lookupCode.getCode()) was used to get the code and if you notice checkPasswords() is then called in this context to test if the fields are the same.

Running this through a browser (with no CSJS yet) we can see a successful test

js4

Then the CSJS is called from the onClick event of the submit button

<xp:button value="Submit" id="button1">
  <xp:eventHandler event="onclick" submit="true" refreshMode="complete" immediate="false" save="true">
    <xp:this.script>
    <![CDATA[
      if (!checkPasswords()){
        alert('passwords must match')
        return false
      }
      ]]>
    </xp:this.script>
  </xp:eventHandler>
</xp:button>

Which looks like this

js5

So why are we doing this again Marky?

Well the point is that CSJS alone is a better experience for the user but it is not secure – so if we can write the validation code once it can be used Client Side and Server Side with the minimum of duplication

Here is how I break the CSJS validation and bypass it – see in firefox – I can override the checkPasswords function to show a different prompt and then return true – submitting the form

js6

Which is then validated using the SSJS – securing the application 🙂

js7

Summary

In both cases we were able to use the checkPasswords() function which was written once and stored in the notes document So as far as I know this is the first example of using the same JavaScript code to validate a form client-side and server-side – which was the initial goal – there is some improvement to be done here though I am sure

Caveats

And there are a few…..

  1. For the uninitiated the eval function is evil and should not be used in production code unless you can absolutely guarantee  the security of the source.
  2. Looking up the code in a document has an overhead and this would not scale well over many functions and many documents
  3. There is a lot more code written to save copying and pasting a few lines of code in this case – this did not make my life in any way shape or form easier
  4. There has to be a better way – and the quest has only just begun

EXTJS in XPages #10 – Grid Row Expander plugin

In this article I will demonstrate how to add expandable and collapsible sections to each grid row using the extjs rowexpander plugin.

EXTJS in XPages series

Here are links to all of the previous articles in this series

Demonstration

The EXTJS in XPages demonstration database can be found at http://demo.xomino.com/xomino/Extjs.nsf/xGridExpand.xsp

Download

The sample database that will accompany this series is  available from the menu in the live demo database from this link – http://demo.xomino.com/xomino/Extjs.nsf

Introduction

Now we have the basics of how to build a grid down I am going to demonstrate some of the cool plugins and features which can be added to the EXTJS grids within our XPages. This article focus on the “rowexpander” plugin which gives us the ability to “expand the row” and format the displayed output.

In the example I have created I have used a normal grid showing 240+ countries and territories in the world. When you click on the row it expands and show the reformatted data fields and the country’s flag.

row1row2

How does it work?

There are no new code bases to be loaded for this plugin it is “out of the box” so to speak. It really could not be simpler…..

I created a new view “vwCountry” and a new REST service to pump out the data

<xe:restService id="country" pathInfo="country">
	<xe:this.service>
		<xe:viewJsonService defaultColumns="true" viewName="vwCountry" count="10000"></xe:viewJsonService>
	</xe:this.service>
</xe:restService>

And then the grid code to retrieve it is the same as the the bufferedRendered example (with new field names)

The significant part is adding the plugin to the grid and inserting the rendering code – but that is also copy and pastable almost. As you can see from the code below there is a ptype: rowexpander plugin and then within that I use an EXT.XTemplate to show the code which I want to display on the screen while the row is expanded. It is as simple as that.

    var grid = Ext.create('Ext.grid.Panel', {
        renderTo: 'gridHere',
        frame: true,
        height: 400,
        layout: 'fit',
        title: 'Countries',
        plugins: [{
            ptype: 'rowexpander',
            rowBodyTpl : new Ext.XTemplate(
                '<p><b>Country:</b> {Name}</p>',
                '<p><b>Life Expectancy:</b> {Life}</p><br>',
                '<p><b>Highest Peak:</b> {HighestPeak}</p>',
                '<p><img class="flag" src="http://www.worldatlas.com/webimage/flags/countrys/zzzflags/{CountryCode:this.lowerCase}large.gif"/></p>',
            {
            	lowerCase: function(cc){
            		return cc.toLowerCase()
            	}
            })
        }],

Conclusion
Once again this shows how quick and simple it is to provide excellent functionality for your users with little to no effort on your part as the developer – and that is really what we all like 🙂

Side note
Originally I used the buffereredRenderer on this grid but there appears to be a display issue (v4.2) which has been reported and tracked with Sencha. I removed the bufferedRenderer and it worked fine.

Someone is having a laugh right?

I got this in my email today – trying to help a friend out

See if you can spot scary………..

 

Role: Lotus Notes Developer

 Location: CHICAGO, IL

Duration: 6+ months

 

Skillet required:

 Experience using and developing applications for Lotus Notes Version 8 Experience using and developing Domino web applications

 Knowledge & experience in Lotus Notes Administration processes

5+ Years experience developing Lotus Notes client applications

Proficiency with the Lotus Notes security model (ACL, ReaderNames, AuthorNames, Domino Directory, etc)

Proficiency with Lotusscript Programming Language

Proficiency with Lotus Notes Formula Language

Great communication & documentation skills

Some knowledge/exposure to Microsoft Excel/Powerpoint

Willing to be available to work 24/7

 Willing to violate US labor & tax laws by working many hours of overtime with no pay

Knowledge of Hindi language would be strategically beneficial

 

If you are interested in this position please respond to me with your resume with following details

Full name :

Visa status :

Contact number :

Best time to reach :

Current location :

Best rate (per hr on w2):

 

Google Chrome 27 – now with added voice searching

The new Chrome v27 went live today with a new voice search capability using the HTML5 media capabilities the browser asks to use my microphone and when it does it allows you to speak your query rather than type it.

It is similar to the capability which has been in Android Jelly Bean 4.2 for a while now which I find very useful and I am becoming more reliant on it – we still have some accent issue to get over though.

Below is a quick video demonstrating that it still has some hurdles to get over and some interesting results 🙂

Kinda my point in highlighting this is really to illustrate the HTML5 media capabilities and where the browser is going. The ability to turn speech into text is not new – but to integrate it with our web applications is…

Writing a single library for SSJS and CSJS validation – failed attempt #1

In this article I will demonstrate my first futile attempt to consolidate CSJS and SSJS libraries so that we only have to write validation once. I will also discuss JavaScripts Closures which unless you have already encountered them will make your head hurt thinking about it 🙂

Introduction

I was reminded in one of the comments on my blog last week about how it is still a pain in the butt to have to write client side JavaScript validation (to provide a good user interface) and server side JavaScript validation (to ensure data integrity and because CSJS validation is easy to break). So I set out to try and simplify this process by trying to copy a basic SSJS JavaScript validation routine in CSJS. I figured if the average XPage developer is now used to writing validation in SSJS then why confuse things. Try and mimic the basic functionality client side and away we go – simple right?

Comparing two fields in SSJS

So here is my basic example – comparing two password fields which have been submitted

<xp:validateExpression message="Passwords Must Match">
   <xp:this.expression><![CDATA[#{javascript:
        if (getComponent("Password").getSubmittedValue() != getComponent("ConfirmPassword").getSubmittedValue())
          {
            return false;
          } else {
            return true;
          }
        }]]>
   </xp:this.expression>
</xp:validateExpression>

Here’s my form in Firefox – the Password field is the one we are going to look at in the examples. I want to test if the Password and ConfirmPassword fields have the same value – in the client and on the server. The above code does it on the server. (originally posted here)

fail1

So the trick is to try and mimic getComponent(“Password”).getSubmittedValue() in client side JavaScript and then the code which is already written can serve to validate on the client side as well – well that is the plan anyway.

Not as simple as you’d have thought

So I started by learning about how to chain JavaScript functions and would you believe it is rather mind-bendingly simple and yet amazing complex to get your head around at the same time.

Reading:

http://stackoverflow.com/questions/9338439/how-to-chain-functions-without-using-prototype/9338486#9338486

http://stackoverflow.com/questions/6256016/how-can-i-call-any-function-in-a-chain-of-functions-without-the-chaining

But the basic premise it to be able to “return this” from the original function which therefore returns a function – which means you can then .doSomething to it.

First – Prototypal inheritance

A cool part of the JavaScript language is that you can create prototype (copy) of an object and it inherits all the traits of the originator – the problem I ran into with this is that you have to create a object in the first place to then chain the function to it – that did not mimic the exact syntax I was looking for – maybe another day. (example from this website)

function Pet(name, species){
  this.name = name;
  this.species = species;
}
function view(){
  return this.name + " is a " + this.species + "!";
}
Pet.prototype.view = view;
var pet1 = new Pet('Gabriella', 'Dog');
alert(pet1.view()); //Outputs "Gabriella is a Dog!"

Then closures

OK for those of you who do not understand JavaScript closures:

  1. Join the club
  2. Sit down with a good cup of tea and a blanket
  3. This is going to hurt

As far as I understand it (and it would not be the first time I was wrong) – a JavaScript closure is basically a function within a function which has access to the locally scoped variables after the function has returned.

In the spirit of Hammer Horror films – a closure is a function within a function which can live on after the original function is finished. It can then feed off the innards of the function which is now dead – nice!

Told you your head would hurt – go read this, it might help http://www.crockford.com/javascript/private.html

So I created the getComponent function like this

var getComponent
getComponent = function(key){
    var temp = dojo.query('[id$=:'+key+']')[0].value
    console.log(temp)
    return this
}

And when I run that you can see we get the value from the field logged and then “window” returned which is the namespace for the functions

fail4
Within the getComponent function I then added the getSubmittedValue()

fail5

Without the need for the console.log – we now get “Marky” returned

How does that work then? You’re right my head hurts……

Yeah me too !

  1. We saw the getComponent function on its own created the temp value and returned itself (making it chainable)
  2. The getSubmittedValue function was then called within the getSubmitted function
  3. getSubmittedValue does not have a parameter passed to it through the originating call
    getComponent(“Password”).getSubmittedValue()
  4. But because it is a closure it does have access to the temp variable which was created when the original getComponent function was called
  5. uuurgh *shakes head*
  6. So anyway it works and woooooo – I got what I wanted !!!
  7. WHY do closures work like this? I haven’t figure that out yet. I think it is cos in this case the getSubmitted function is still in memory even though it has been “completed” it still exists and can therefore still be “used” – but that is just a  guess on my part.

This then allows be to create a generic checkPasswords function which should run with the same “JavaScript” for client side or server side – man I was dancing around the room after spending the entire week trying to figure this out 🙂

var getComponent
getComponent = function(key){
    var temp = dojo.query('[id$=:'+key+']')[0].value
    getSubmittedValue = function(){
        return temp
    }
    return this
}

function checkPasswords(){
	if (getComponent("Password").getSubmittedValue() != getComponent("ConfirmPassword").getSubmittedValue()){
      return false;
    } else {
      return true;
    }
}

checkPasswords()

Implementation
Then I tried to implement it in my XPage – and got the face palm moment – ServerSide JavaScript Libraries and Client Side JavaScript libraries are discrete and nether the two shall cross…..

You cannot include an SSJS library on your “web page” and you cannot include a CSJS library in your Server Side code

SON OF A BISCUIT EASTER !

Conclusion

So there’s my fail – but even after elation and then facepalm I am not down about this at all.

I am actually very happy about how much I heave learned about JavaScript this week – I have always considered myself “ok” at JavaScript but never advanced enough to start to write it as a “JavaScript developer”. I think that path has started and I am going to be writing about it I am sure.

If anyone has any suggestions or insight on how I can possibly move this forward please let me know. I would love to be able to create a single method for doing SSJS and CSJS validation without the need for duplicative coding.

🙂

XPages ND9 (Dojo 1.8) does not work with other AMD loaders (and the work around)

Today I took a new jQuery plugin that I was playing with from an R8.5.3 environment where it worked, to an R9 environment and it failed – I figured it had something to do with something I had been reading about over the last month and in fixing my problem this lead to a more serious realization that I KNOW other people are also going to come across.

Background

A brief scan of the last month’s discussions on Xpages and jQuery Mobile 1.3 reveals a number of posts started by Dave Leedy regarding the use of IBM Notes Domino 9 and jQuery Mobile 1.3

http://notesin9.com/index.php/2013/04/20/jquery-mobile-seems-incompatible-with-xpages-in-domino-9-0

http://hasselba.ch/blog/?p=1216

http://stackoverflow.com/questions/16058039/jquery-mobile-in-xpages-passing-parameters

http://www.eknori.de/2013-04-21/followup-jquery-mobile-seems-incompatible-with-xpages-in-domino-9-0/

What’s going on?

I saw the following Tip from Sven Hasselbach (very smart man!) on Dave Leedy’s blog post

Looks like an AMD loader problem with Dojo & JQM. I have tried out to disable the Factory scan in a test page and this worked for me. To do this I have added the following parameters in the xsp.properties:

xsp.client.script.dojo.djConfig=dojo-amd-factory-scan: false

Maybe this can help you with your problems in ND9.

This lead me to this bug report

https://bugs.dojotoolkit.org/ticket/15616

make dojo work with other AMD loaders

Which then lead me to the realization that this is a bigger issue than just JQM 1.3 and I solved my problem and here is how.

Solution (ND9)

Within the Package Explorer  (Window—open perspective –> xpages) in the WebContent folder for your database you will find the xsp.properties file (as described http://heidloff.net/home.nsf/dx/06172011015521AMNHE8Y7.htm)

xsp1

Click to open the Source Tab

and enter the following

xsp.client.script.dojo.djConfig=dojo-amd-factory-scan: false

And Save – that’s it

Result

If you look at the source code of the created XPage you will see that the new parameter has been added to the script tab creating the dojo.js as a djConfig property

xsp3

doing a quick search explains what this property does

http://dojotoolkit.org/documentation/tutorials/1.8/dojo_config/

“For example, we could disable the amd factory scan (scanning the module for CommonJS require(module) statements to load as deps) with…”

Not only does that solve the problem with jQuery Mobile, it also solved my problem with the jQuery Table plugin which I wanted to use, which used require.js to load the jQuery dataTable plugin.

GENIUS

thank you Sven Hasselbach (again!)

Server-side HTML vs. JS Widgets vs. Single-Page Web Apps – the XPages version

Yesterday I came across this excellent article by Pamela Fox – http://blog.pamelafox.org/2013/05/frontend-architectures-server-side-html.html. In it she goes through how her company uses all three stated architectures, discusses why, and how they are used by the end users. It struck me as fascinating because I feel like we are going through this exact same struggle in the XPages community – what is the best architecture to create applications for our users? I also feel fortunate to have experienced all three (to an extent) in the past 18 months and I can empathize/understand Pamela’s perspective.

I am writing this blog article to first highlight the original article, but to also put an XPages spin on it and discuss some of the point raised therein. I have been meaning to write on this subject for a while and Pamela’s article said it better than I could because she has real examples to discuss whereas mine was all hypothetical.

Please read the article  before continuing, otherwise the context would be rather lost 🙂

Server-side HTML (“Web 1.0”)

“This architecture suffers the most in terms of usability – it’s very hard for users to do many interactions in a small amount of time – but it does have definite benefits of easy linkability, shareability, and searchability.”

XPages out of the box is a technology built on this paradigm. The tools provided make it quick and easy to create functional applications based entirely on a request-response constant back and forwards to the server. The bulk of the application logic is based on the server and that is very much akin to traditional Lotus Domino development. XPages does at least move Lotus Domino in the right direction, in that we now have partial refresh and the programmatic ability to execute Server-Side logic without having to completely reload the page.

JS widgets

This is something I really want to get into more. If you look at backbonejs or other JavaScript modelling libraries which model data I think you will see that they have a lot of architectural similarity to Lotus Domino. Flat Data is managed in views(notes document)  models(notes forms) and collections (notes views) except that the bulk of the logic is performed Client-Side. Backbone is based on the premise that REST is the communication medium to get data and update it on the back end. We have REST services in ExtLib and in R9 Data Services out of  the box.

In many senses this is more work than creating a traditional XPages/Notes Domino application because the developer has to code everything by hand – there are no (IBM XPages provided) tools to help develop this architecture.

But the payoff to the end user is significant – the data transfer is as minimal as possible and the transactions with the server are fast – ultimately leading to a faster user interface.

As the application complexity scales, so does the amount of work (and code) necessary to make this happen. This also raises a question about maintainability across an enterprise.

Single-page web apps

With the introduction of the ExtLib Dynamic  Content control Single-page web applications are a real possibility within XPages. The user never has to leave the “XPage” to be able to interact with their entire application. With the controlled use of partial refresh the user can move from View Panel, to Document and back again without having to reload the entire page…ever.

This lead to a very consistent, smart, user interface.

The difference between a corporate web application and a “web-site” become more apparent though as the size and complexity of the application increases.

Twitter is a single-page application – but the overall front end user interface features are relatively limited. Maybe 5-10 different screens and most of those are data reading only – the amount of “update” is limited to posting tweets and profile update.

In a corporate application you could have a significant number of “forms” which require update and an equally heavy number of data views. The complexity for keeping this all on one page does not scale well.

Another consideration is the amount of script libraries etc you have to load when the user first accesses the application. More significant if you have to support older IE browsers or XPiNC which cannot handle dynamic script injection via AJAX. If you have 100 “pages” and only one of them uses a specific plugin (org chart for example) you have to load that script library, regardless of whether or not the user is going to access that page or not. That’s kinda nasty.

So what is best for XPages?

I believe that all development decisions should be grounded in the user experience, not on the easy of development. For too long Lotus Notes has succeeded and then failed by allowing people to build functional applications quickly (cheaply) and then having other people mock it for “working” but looking like crap and/or being too “hard” to use.

That said there is a definite bottom-line balance between the $500,000 application overhaul to code everything in the browser and make an amazing application, and the $100,000 application which works great, is maintainable by the in house development team and if 80% awesome on the end user. The builders of amazing Internet websites do not necessarily have the same restrictions and decision makers as “corporate developers“.

So I think the answer is a balance – if you have a large application broken into 5 functional areas, then why not have a widget or single page like application for each functional area?

The point is that there are many factors which go into designing the application architecture from the ground up – and we *ALL* need to understand the *ALL* options so that we can make the right decision for the customer and end users.

Please – discuss 🙂

DCLUG – May 23rd – IBM Sametime Deployment – A look at where are we and where should we go

This month we have Maurice Cogdell speaking to us about his recent experiences with Sametime and where it is going in the market.

If you are to attend this meeting YOU MUST use the meetup site (listed below) to state you are attending – name badges will be created prior to the meeting as your access.

Date

23rd May 2013

Address

The meeting will be held at

IBM Technical Exploration Center
401 Greensboro Drive, McLean, VA

Agenda

————————————————————-

11:30am-12:00pm
Networking (lunch provided by IBM)
12:00pm-12:30pm
Welcome
Community Updates
Show and Tell (Group quick shares)
Tip of the month
Announcements for the next meeting

12:30pm – 1:15pm – Presentation

Maurice Cogdell (IBM) – IBM Sametime Deployment – A look at where are we and where should we go

In this session we will take a look at where we are with IBM Sametime, how to get started, and how to successfully deploy the components you require.

IBM Sametime’s modular design allows you many deployment options and complete flexibility. “We will give examples of the key questions that need to be addressed and common responses, that are used to determine which features you want and need…”.

1:15pm-1:30pm
Questions/Answers Networking
1:30pm
END

Please mark the date in your calendar.
If you have any questions please do not hesitate to contact me.

Mark

More information

For more information and to get involved with the DCLUG

http://www.meetup.com/DC-Lotus-Professionals/events/118427022/

EXTJS in XPages #9 – Infinite scrolling rebooted and reborn – v4.2 BufferedRenderer

In this article I will introduce the new EXTJS v4.2 Infinite scroller – the BuffererRenderer. The whole concept of infinite scrolling has been rewritten in the new version of the grid and it had made a huge difference to responsiveness and stability of the infinite grid.

EXTJS in XPages series

Here are links to all of the previous articles in this series

Demonstration

The EXTJS in XPages demonstration database can be found at http://demo.xomino.com/xomino/Extjs.nsf/xBufferedRenderer.xsp

Download

The sample database that will accompany this series is  available from the menu in the live demo database from this link – http://demo.xomino.com/xomino/Extjs.nsf

Introduction

The first thing I suggest you do is go and watch this blog post and video – it explains in WAY more detail that I possibly could how cool the new grid is.

Blog: First Look at Ext JS 4.2 Grid

Video: http://vimeo.com/59611158

And for those of you who couldn’t be bothered the upshot of all of this is that the grids are WAY better than ion 4.2. The synopsis from my perspective is:

  • 10,000 records are loaded into memory and only a fraction of them are displayed in the grid. As soon as a new row is scrolled into view and added to the grid’s DOM model – a previous row is removed. This makes for a small DOM and takes up a finite amount of browser memory.
    • The increase in XPiNC responsiveness is night and day – it is that much better
  • We no longer need remote filtering and sorting
    • thank goodness cos that was a pain and not as flexible as I wanted it to be anyway
  • We can easily take advantage of grouping a large number of records efficiently
  • Selecting multiple documents across the infinite scroller is retained (HUGE DEAL)
  • Grouping is seamless across the infinite scrolling
  • The only downside is that you have to wait for 10,000 records to load from the REST service – and that can be optimized as I have previous discussed
    • This also means we need to be smarter in reloading the grid only when we really need to
    • If it is a 100k download and takes 10 seconds to render – the user will wait the first time but not every time they need to open a document and then come back to the grid because it was not the one they wanted.
  • In some ways Buffered Render is not really an infinite scroller – because it does not go back to the server ever time the user needs to scroll down – it is really just a highly efficient document display modeler I guess.

So how do we make it work

Well there are some changes to our code but not too many and all of these should be familiar concepts based on the rest of the posts in this series so far.

First  we need to load the data into memory – we do this by calling a REST service. We are going to use a different REST service as well. In the previous infinite scroll example we needed the REST service to tell us how many documents there were in the remote store – we do not need that any more as we are going  to load every document in the view. We are going to use a flat JSON ViewService to pull all the documents from the ByFirstName view

The new REST Service

<xe:restService id="byFirstNameFlat" pathInfo="byFirstNameFlat">
	<xe:this.service>
		<xe:viewJsonService defaultColumns="true" count="10000&" viewName="byFirstName">
		</xe:viewJsonService>
	</xe:this.service>
</xe:restService>

Starting the process
The data is loaded in a different way for the bufferedRenderer. Instead of being loaded from a remote service it is loaded from a local data model. That is not a problem for us but means an extra step in the data management process.

The process kicks off in the following manner

Ext.onReady(function(){

	//We are going to pass to the getDataGrid function the url we want
	//and the callback function which we then wants executing once the data is retrieved
	//doing it this way gives us maximum control over what happens to the data
	//and makes the function more generic should we want to use it for other data
	//and other grids in the future

	var url = 'xRestService.xsp/byFirstNameFlat?count=100000',
			theData
	var theCallBack = function(theData){
			createGrid(theData)
			}
	getGridData(url, theCallBack)
});

In this function we “getGridData” and then passing in a callback function we determine what happens after the data is loaded. We are passing in the function variable theCallBack. This allows the getGridData fucntion to execute on what was passed in to it.

I cannot streaa enough how much of a good JavaScript coding practice this is. Instead of hard coding the getGridData function to call the next step in the process – we tell it what to do.
This creates a generic getGridData function which does nothing but that. It can then be used to do anything else we want – because we are able to tell it what to do next.

Check out my article on Make your XPages more maintainable – JavaScript Callback functions for a better understanding on how callback functions work.

function aRandomNumber() {
	return Math.floor(Math.random()*1000001)
}

function getGridData(url, callback){

	var dataURL = url
	$.ajax({
		  url: dataURL+&quot;&amp;rand=&quot;+aRandomNumber()+&quot;&amp;&quot;
		}).done(function ( data ) {
			//one the data is loaded then pass it to the callback function
			//passed in as one of the function arguments
			if(callback){
				callback(data)
			}
		});
}

Loading the data

$.ajax({
	url: 'xRestService.xsp/byFirstNameFlat'
	})
	.done(function ( data ) {
		createGrid2(data)
	});

The grid code has an additional feature for bufferedrenderer. This can either be added as a text string property in the grid or as a feature object of type bufferrenderer. If we do it this second way we have more control over its properties and it separates it from the main grid code.

buffRend = new Ext.grid.plugin.BufferedRenderer( {
        pluginId: 'buff1'
})

The grid
The grid has the renderer as a grid property.

&lt;strong&gt;The store&lt;/strong&gt;
	store = new Ext.data.Store({
		model : 'Person',
		autoLoad: true,
		sorters: {
			        property : 'firstName',
			        direction: 'ASC'
			    },
		id: 'store',
		data: data,
		remoteFilter: false,
		buffered: false,
        proxy: {
            type: 'memory'
        }
	});

The Grid
The grid itself is not that different from the previous examples except for the one critical line – plugins: buffRend

The BufferedRendered is added as a plugin by first defining the Ext.grid.plugin.BufferedRenderer and then adding it as a property to the grid during creation.

 var filters = {
        ftype: 'filters',
        // encode and local configuration options defined previously for easier reuse
        //encode: encode, // json encode the filter query
        local: local   // defaults to false (remote filtering)

	};

buffRend = new Ext.grid.plugin.BufferedRenderer( {
     pluginId: 'buff1'
})

	gridDock = [{
		id: 'activeStorePaging',
	     xtype: 'toolbar',
	     dock: 'bottom',
		 items: [
		   {
		       text: 'Clear Filter Data',
		       handler: function () {
		           grid.filters.clearFilters();
		       }
		   }
	    ]
	 }]

    var grid = Ext.create('Ext.grid.Panel', {
        renderTo: 'gridHere',
        frame: true,
        height: 400,
        title: 'Users',
	plugins: buffRend,
        features: [filters],
        store: store,
        iconCls: 'icon-user',
        columns: [{
            header: 'First',
            sortable: true,
            dataIndex: 'firstname',
            filterable: true
        }, {
            text: 'Last',
            sortable: true,
            dataIndex: 'lastname',
            field: {
                xtype: 'textfield'
            },
            filterable: true
        }, {
            text: 'Address',
            sortable: true,
            dataIndex: 'address',
            field: {
                xtype: 'textfield'
            },
            filterable: true
        }, {
            text: 'City',
            sortable: true,
            dataIndex: 'city',
            field: {
                xtype: 'textfield'
            },
            filterable: true
        },{
            text: 'State',
            width: 80,
            sortable: true,
            dataIndex: 'state',
            field: {
                xtype: 'textfield'
            },
            filterable: true
        }],
        // paging bar on the bottom
        bbar: gridDock
    });

}

Performance
If you watched the video you will see how the improvements in performance are handled. In the example shown in the video they show that for a few thousand records there are hundreds of thousands of DOM elements created for an old flat grid to be scrolled through. Using the new bufferedRenderer there are always the same number loaded – more in the order of 2,000 regardless of the overall data size. This is acheived by programmatically adding an removing the table rows to the visible grid as the user scrolls up and down the grid. If the user can see 25 rows then only 27 rows are loaded. The grid Tbale displaying the data is constantly modified as the user scrolls up and down.

The difference between 4.1.1 and 4.2 is night and day. I don’t just say that lightly. In the latest application we were working on, with 4.1.1 we were seeing a slowdown in XPiNC around 200 documents in the view. With 4.2 we have loaded over 3000 documents without any significant reduction in responsivness and/or speed of the grid.

Night and Day – it is a different application to interact with – quite remarkable!!

Conclusion
Looking at this demo alone on a modern browser you would not really notice much of a difference – but if you compare some of the other examples in XPiNC to this one you will really notice the improvement. That larger the data set the better then improvement.