Sorting an unsorted XPages document collection based on a date field

In one of my recent applications I had a requirement to return the top 500 search results, but sort them by a specific date field on the document, in descending order. Due to the size of the database doing a view.FTSearchSorted was not feasible as it was too slow. So I searched around and found some code for sorting a JavaScript Array of objects.

Creating the array

Search the database and push the search results into a JavaScript array

var sSearch = "[FIELD Subject contains smith]"
var dc:NotesDocumentCollection = db.FTSearch(sSearch, 500)
var doc:NotesDocument = dc.getFirstDocument();
var myArray=[]
var i=0
var sTemp = ""
while (doc != null) {
	i=i+1
	var obj={}
	obj.count=i
	obj.Customer = doc.getItemValue('Customer')
	obj.malfunctionEnd = @Text(doc.getItemValue('dateField'))
	obj.unid = doc.getUniversalID()
	myArray.push(obj)
	var tmpdoc = dc.getNextDocument();
	doc.recycle();
	doc = tmpdoc;
}

The sort function

Create a function to return a bubble sort of dates. This function works by comparing if two dates should be sorted based on their “getTime()” which returns a value in seconds.

function custom_sort(a, b) {
var ndtA:NotesDateTime = session.createDateTime(a.dateField);
var ndtB:NotesDateTime = session.createDateTime(b.dateField);

return ndtB.toJavaDate().getTime() - ndtA.toJavaDate().getTime()
}

The final step

Using the JavaScript sort function we can pass in the name of a callback function which adds additional processing to the sort function. The resulting array can then be processed by a repeat control providing the information out to the end user. All the code shown here would be placed in the value parameter of the repeat control.

myArray.sort(custom_sort);
return myArray

11 thoughts on “Sorting an unsorted XPages document collection based on a date field

  1. From https://github.com/OpenNTF/org.openntf.domino/tree/nathan/org.openntf.domino

    String sSearch = “[FIELD Subject contains smith]”;
    org.openntf.domino.DocumentCollection dc = db.FTSearch(sSearch, 500);
    List criteria = new ArrayList();
    criteria.add(“datefield”);
    org.openntf.domino.helpers.DocumentSorter sorter = new org.openntf.domino.helpers.DocumentSorter(dc, criteria);
    org.openntf.domino.helpers.DocumentCollection sortedColl = sorter.sort();

    http://en.wikipedia.org/wiki/Bubble_sort#Performance

  2. From https://github.com/OpenNTF/org.openntf.domino/tree/nathan/org.openntf.domino
    String sSearch = “[FIELD Subject contains smith]“;
    org.openntf.domino.DocumentCollection dc = db.FTSearch(sSearch, 500);
    List criteria = new ArrayList();
    criteria.add(“datefield”);
    org.openntf.domino.helpers.DocumentSorter sorter = new org.openntf.domino.helpers.DocumentSorter(dc, criteria);
    org.openntf.domino.helpers.DocumentCollection sortedColl = sorter.sort();
    http://en.wikipedia.org/wiki/Bubble_sort#Performance

    • On an unrelated issue:

      By combining org.openntf.domino with the jvm language Groovy (which works perfectly on Domino), you can write code like this to sort.
      (Not FTSearch, but you get the idea). As long as the classes return an iterator() (and some now do that, yay!), you can use the default Groovy collection mixins, i.e. each, collect, sort, find, findAll.

      def items = []
      items = db.search(‘[FIELD Subject contains smith]’).sort { Document doc ->
      doc.getItemValue(‘SomeDateField’)[0].toJavaDate()
      }

      Loop a view, get all documents, use unid as key in a Map:

      def eomployees = [:]
      nab.getView(‘($People)’).getAllDocuments().each { Document person ->
      employees[person.getUniversalID()] = person
      }

      Create and save a document:

      db.createDocument([
      Form: ‘Person’,
      Type: ‘Person’,
      MasterDocId: person.getUniversalID(),
      SyncID: person.getUniversalID()
      ]).save()

      It’s almost pretty now😉

  3. Don’t forget to recycle your DateTime instances too… you’re leaking two handles in every call to custom_sort().

    Also, don’t forget to recycle the DocumentCollection when you’re done with it as well.

    Less significant, but potentially problematic: every line that doesn’t end with a curly brace should end with a semicolon. Otherwise your whitespace is meaningful, which can cause accidental removal of a single carriage return to break the entire function / library. Inclusion of semicolons ensures that whitespace assists legibility but has no impact on runtime behavior. In client-side JavaScript, omission of semicolons can actually slow execution of the code because of the way the parser has to interpret the whitespace to guess where the semicolons should have been… to be honest, I haven’t done any benchmarking to determine whether this performance degradation occurs within SSJS as well, but if IBM is truly faithful to the ECMAScript spec (aside from their custom uses for @ and :, of course), then it would have at least some impact… although not enough to be noticeable until the application tries to scale.

    Finally, custom_sort() is referring to a dateField property of each array member, but you’re never setting that property on the objects you’re adding to the array… I’m assuming you would want to sort by the malfunctionEnd property, since you’re assigning that property the value of the dateField item on each document (related: you can skip the call to @Text if you use getItemValueString instead of getItemValue).

    Sorry for the barrage…😉

  4. My SSJS is a bit rusty, but in Java I would use a TreeMap, put the date (in it’s long format) as a key, and allow the Map to sort itself.

    Pseudocode:
    var sSearch = “[FIELD Subject contains smith]”
    var dc:NotesDocumentCollection = db.FTSearch(sSearch, 500)
    var doc:NotesDocument = dc.getFirstDocument();
    var sortedElems:Map = new java.util.TreeMap();
    var i=0
    var sTemp = “”
    while (doc != null) {
    i=i++
    Date leDate = doc.getItemValue(‘dateField’).toJavaDate();
    var obj={}
    obj.count=i
    obj.Customer = doc.getItemValue(‘Customer’)
    obj.malfunctionEnd = @Text(doc.getItemValue(‘dateField’))
    obj.unid = doc.getUniversalID()
    sortedElems[leDate.getTime()+i] = obj; // add the counter to make sure the time is unique

    var tmpdoc = dc.getNextDocument();
    doc.recycle();
    doc = tmpdoc;
    }

    return sortedElems.values(); // returns the Collection in sorted order

    • You could also do this with a TreeSet and a Comparator like:

      public class DocDateComparator implements Comparator {
      private final String fieldName_;

      public DocDateComparator(String fieldName) {
      fieldName_ = fieldName;
      }

      public int compare(Document docA, Document docB) {
      try {
      DateTime dtA = (DateTime)docA.getItemValue(fieldName_).get(0);
      DateTime dtB = (DateTime)docB.getItemValue(fieldName_).get(0);
      int result = dtA.toJavaDate().compareTo(dtB.toJavaDate());
      dtA.recycle();
      dtB.recycle();
      return result;
      } catch (NotesException e) {
      // Argh.
      e.printStackTrace();
      return 0;
      }
      }
      }

      Then build the TreeSet like new TreeSet(new DocDateComparator(“SomeFieldName”)). Of course, if you’re using the API, this would probably be largely comparable to the DocumentSorter Nathan mentioned.

  5. Guys – LOVE the feedback thank you *so* much – this is what helps to make a useful resource for the community and I really appreciate it🙂

    I will try these examples and see if there is any perceptible increase in speed of results return. If there is expect to see the results soon🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s