In this article will demonstrate how simple planning and insight can make your XPages EXTLib REST service more efficient.
Problem
Whatever you are using your REST service for, wither it is to feed charts, grids, other application or whatever it is, they take up data bandwidth and processing time.
Consider the following notes view
If I display that in a simple REST service one of the entries looks like this:
{
"@entryid":"1-45995DDEB92AA70C882579F0008280CB",
"@unid":"45995DDEB92AA70C882579F0008280CB",
"@noteid":"1A12",
"@position":"1",
"@siblings":200,
"@form":"Contact",
"$13":"Adriana Kerr",
"FirstName":"Adriana",
"LastName":"Kerr",
"EMail":"adriana_kerr@renovations.com",
"City":"Norwalk",
"State":"CA",
"Created":"2012-04-30T23:45:26Z",
"$10":"Adriana Kerr"
},
This batch of data takes up 462 bytes. The first 6 lines the attributes @entryid to @form you have no control over. But you do have controll over the others.
The field name displayed comes from the computed column value in the view. If you have a column which is just a single field the programmatic name will reflect the field Name.
But if you have a column value which not just a fieldname then you will have a programmatic id assigned
1) Taking control of the column names
The point is that these values can be changed
And in doing this your rest service changes:
{
"@entryid":"1-45995DDEB92AA70C882579F0008280CB",
"@unid":"45995DDEB92AA70C882579F0008280CB",
"@noteid":"1A12",
"@position":"1",
"@siblings":200,
"@form":"Contact",
"1":"Adriana Kerr",
"2":"Adriana",
"3":"Kerr",
"4":"adriana_kerr@renovations.com",
"5":"Norwalk",
"6":"CA",
"7":"2012-04-30T23:45:26Z",
"8":"Adriana Kerr"
},
And this batch of data takes up 426 bytes (as opposed to 462 before). This right there represents a saving of 8% which might not seem like much but this is only a small view with only a few columns with only short field name.
If you scale this up to a large view with 25 columns and large field names that can make 2000 bytes per entry – I just performed a quick test on one of my larger REST services and saved over 25% of the data (which is what inspired me to write the article).
“Soooo what” I hear you say – well consider 500 data entries in my world. That ramps up very quickly to a 1M data transfer – saving 25% on that is a significant bandwidth saving – not to mention the reduction in load on your browser having to parse that amount of data.
XPiNC memory
This is especially pertinent in the XPiNC world which I have the unfortunate pleasure of playing with – it is Firefox 3.5 and has memory issues. I have nothing quantifiable to back this up but there is no doubt about it in my mind – a large data volume causes issues in the browser memory management. This is highlighted by a slowdown of the interface – you know the things we USED to have a problem with but don’t any more with Chrome and Firefox browsers.
2) Better control and only send what you need….
This should be common sense but if you are upgrading an existing database and want to take advantage of existing views, might not always be apparent.
By default the view send all the columns when you select defaultColumns=”true”
<xe:restService id="restService3" pathInfo="contactjson"> <xe:this.service> <xe:viewJsonService viewName="AllContacts" var="entry" contentType="text/plain" count="10" defaultColumns="true"> </xe:viewJsonService> </xe:this.service> </xe:restService>
By going into the all properties and selecting the columns you want you have more control and save bandwidth/memory management
In this case I put the column names back and programmatically added the columns to the REST service
And then I added the id and firstName column programmatically to the REST service – but as you can see I forced the name of the field in this case to be 1 and 2
<xe:restService id="restService3" pathInfo="contactjson"> <xe:this.service> <xe:viewJsonService viewName="AllContacts" var="entry" contentType="text/plain" count="10" defaultColumns="false"> <xe:this.columns> <xe:restViewColumn name="1" columnName="id"></xe:restViewColumn> <xe:restViewColumn name="2" columnName="firstName"></xe:restViewColumn> </xe:this.columns> </xe:viewJsonService> </xe:this.service> </xe:restService>
This creates the optimized JSON feed with the smaller field name, without changing it at the column.
{
"@entryid":"1-45995DDEB92AA70C882579F0008280CB",
"@unid":"45995DDEB92AA70C882579F0008280CB",
"@noteid":"1A12",
"@position":"1",
"@siblings":200,
"@form":"Contact",
"1":"Adriana Kerr",
"2":"Adriana",
},
This code is (for example) all that is needed from this view and at 263 Bytes it is 39% smaller than the original
Caveats
- If the bulk of your data is in the field value and not in the field names you will not see such a large improvement.
- Changing the column names in a view needs to be done carefully as those names could be referenced elsewhere.
- When you have a $10 programmatic column name and you change the formula in the column the name is recomputed – be aware that this can change
Xomino 
Is Adriana Kerr the secret clone daughter of Adriana Lima and Miranda Kerr? Because if so, can you please display a phone number column as well? I mean, seriously; playas gotta play.
Interesting but I would say this is only useful if there’s only 1 developer and you weren’t intending to leverage the API somewhere else – say passing to another Web Developer.
It would drive me potty to try and work with that JSON data, imagine using the dot notation syntax to get to the first name.
BTW I think It is possible to remove the programmatic entries but you have to write your own custom Java beans / OSGI plugin which extends the inbuilt REST services.
I did some tests with GZiped JSON Data against AMF, which is a binary format, and it worked out pretty close for reasonably large sets of data so it might be worth checking your Domino server has GZip turned on.
We create the REST APIs for our applications manually by selecting/filtering view data (e.g. all entries, subcategory navigator or ft search result collection), traversing view entries (leveraging the ViewNavigator caching if possible) and writing the JSON data to the servlet output writer with an improved version of the org.json.JSONObject classes or Jackson JSON processor.
I just like to know what’s going on in the code and reinventing the wheel is my hobby ;-).
I’m wondering how they retrieve “@form” from the view. The last time I checked the Extlib code, it was accessing the document to read this field, which reduces performance a lot. I think there was a comment in the code that this should be removed or fixed asap.
Interesting indeed as Mark Barton already noted but I really would hate to see a rest service which returns data in fields with names like 1 , 2 or 3. Names of variables should always, in any case, tell the developer what he can expect from it. When using numbers as names developers have no clue what the data is used for and therefore can build unwanted results.
So that tip is actualy not a tip. I would even call it a little bug you are introducing. It would be best to see if you can somehow compact the data another way such as generating only the columns you need ( as you already pointed out).
I am sure there is a balance of usability and data overload. If this is a public REST service I agree field names might be best – but for an internal application which is suitably documented in the appropriate place maybe LN would work for LastName.
In my case I had fields as long as txtLastKnownAddress and what’s even worse when the field is blank in the service
I believe LKN would be sufficient in this case and be 80% smaller
Maybe my example is an extreme case but highlights to possibility and considerations
Found it in the Extlib code (version 853.20121217-1354) in class com.ibm.domino.services.rest.das.view.RestViewNavigatorFactory:
// Added by Tim Tripcony
// Hoping to add support for Form Formulas later on
@Override
public String getForm() throws ServiceException {
String result = “”;
if (isDocument()) {
try {
Document document = entry.getDocument();
if (document != null) {
result = document.getItemValueString(ITEM_FORM);
}
} catch (NotesException ex) {
throw new ServiceException(ex,””); // $NON-NLS-1$
}
}
return result;
}
They fetch the entry document, but there is no recycle call afterwards. Have you tried calling your REST services with many documents, e.g. several thousands? Might lead to a crash, because Notes is running out of handles.
For shame… what was that guy thinking?
🙂
Seems to be fixed in the latest extension library release 853.20130215-1516.
No, not fixed. Looked at the wrong class. Still there
You can also specify which system columns (the ones with @ as prefix) to return by using the systemColumns property of the REST Service.
Noticed a slightly reduced loading time.
I also find using viewItemFileService faster than using viewJsonService.
also, setting compact = true in the REST Service will reduce the size some.. In my case about 10%
oo thank you – really appreciate that 🙂
jeux habillage monster high jeux de mario bros 3 gratuit
I know I am late to this party but there’s another setting people might find useful. If you set systemColumns = 0, all those @Siblings, @Read, @Position … values will be gone. You can save a nice chunk of bandwidth right there.
thanks mate 🙂
How to format date here with some other separator. Currently it display as “2012-04-30T23:45:26Z”.
If i need separator to be “/” instead of “-“,how to achieve that.