jQuery in XPages #8 – Tokeninput autocomplete

In these two articles I am going to illustrate and describe how to implement an alternative to the XPage typeAhead, using the “jQuery Tokeninput“. Unlike the XPages typeahead the jQuery Tokeninput has a large and flexible paramaterized API which allows for control over how the results are displayed, how the results are created and the source of the data. We are going to look at how to search a basic JSON string, a notes agent generated JSON feed, a twitter JSON feed, and an XPage REST JSON feed.

Tokeninput.js

Introduction
Tokeninput provides a typeAhead-like capability whereby a user is presented with matching options as they start to type into a text field. The source of the data can either be a local JSON object or a web provided JSON object.

Basic Tokeninput user interaction
Tokeninput with facebook look and feel
Tokeninput with facebook look and feel
Tokeninput with mac look and feel
Tokeninput with mac look and feel

The search results are determined from a JSON string and can come from any web source; flickr, twitter, facebook and domino to name but a few examples. What is really nice about Tokeninput is the control and flexibility provided to the developer out of the box allows you to make some very impressing, interactive webpages with very little effort. Unlike the XPage typeahead Tokeninput will also allow you to select multiple values and the display of these values can be controlled using a CSS.

Example Twitter feed with formatted results
Example Twitter feed with formatted results

Demonstration

The XPages integration of Tokeninput is demonstrated here

Download

The demonstration database can be downloaded from the link above or from here

How does it work in XPages?

For a more detailed dive into how the examples can be recreated and function in an XPage please check out the how does it work article

Tokeninput API

Tokeninput is extremely versatile and has a large number of parameters (as of v1.6) – these are taken straight from the website and you should check out all the examples on the website to get a good idea of all the capabilities

Search Settings

  • method
    • The HTTP method (eg. GET, POST) to use for the server request. default: “GET”.
  • queryParam
    • The name of the query param which you expect to contain the search term on the server-side. default: “q”.
  • searchDelay
    • The delay, in milliseconds, between the user finishing typing and the search being performed. default: 300 (demo).
  • minChars
    • The minimum number of characters the user must enter before a search is performed. default: 1 (demo).
  • propertyToSearch
    • The javascript/json object attribute to search. default: “name” (demo).
  • jsonContainer
    • The name of the json object in the response which contains the search results. This is typically used when your endpoint returns other data in addition to your search results. Use null to use the top level response object. default: null.
  • crossDomain
    • Force JSONP cross-domain communication to the server instead of a normal ajax request. Note: JSONP is automatically enabled if we detect the search request is a cross-domain request. default: false.

Pre-population Settings

  • prePopulate
    • Prepopulate the tokeninput with existing data. Set to an array of JSON objects, eg: [{id: 3, name: “test”}, {id: 5, name: “awesome”}] to pre-fill the input. default: null (demo).

Display Settings

  • hintText
    • The text to show in the dropdown label which appears when you first click in the search field. default: “Type in a search term” (demo).
  • noResultsText
    • The text to show in the dropdown label when no results are found which match the current query. default: “No results” (demo).
  • searchingText
    • The text to show in the dropdown label when a search is currently in progress. default: “Searching…” (demo).
  • deleteText
    • The text to show on each token which deletes the token when clicked. If you wish to hide the delete link, provide an empty string here. Alternatively you can provide a html string here if you would like to show an image for deleting tokens. default: × (demo).
  • animateDropdown
    • Set this to false to disable animation of the dropdown default: true (demo).
  • theme
    • Set this to a string, eg “facebook” when including theme css files to set the css class suffix (demo).
  • resultsFormatter
    • A function that returns an interpolated HTML string for each result. Use this function with a templating system of your choice, such as jresig microtemplates or mustache.js. Use this when you want to include images or multiline formatted results default: function(item){ return “<li>” + item.propertyToSearch + “</li>” } (demo).
  • tokenFormatter
    • A function that returns an interpolated HTML string for each token. Use this function with a templating system of your choice, such as jresig microtemplates or mustache.js. Use this when you want to include images or multiline formatted tokens. Quora’s people invite token field that returns avatar tokens is a good example of what can be done this option. default: function(item){ return “<li><p>” + item.propertyToSearch + “</p></li>” } (demo).

Tokenization Settings

  • tokenLimit
    • The maximum number of results allowed to be selected by the user. Use null to allow unlimited selections. default: null (demo).
  • tokenDelimiter
    • The separator to use when sending the results back to the server. default: “,”.
  • preventDuplicates
    • Prevent user from selecting duplicate values by setting this to true. default: false (demo).
  • tokenValue
    • The value of the token input when the input is submitted. Set it to id in order to get a concatenation of token IDs, or to name in order to get a concatenation of names. default: id

Callbacks

  • onResult
    • A function to call whenever we receive results back from the server. You can use this function to pre-process results from the server before they are displayed to the user. default: null (demo).
  • onAdd
    • A function to call whenever the user adds another token to their selections. defaut: null (demo).
  • onDelete
    • A function to call whenever the user removes a token from their selections. default: null (demo).
  • onReady
    • A function to call after initialization is done and the tokeninput is ready to use. default: null


Advertisements

jQuery in XPages #8 – Tokeninput autocomplete – how does it work?

This article is written in support of the original jQuery in XPages #8 – Tokeninput autocomplete article

Demonstration

The XPages integration of Tokeninput is demonstrated here

Download

The demonstration database can be downloaded from the link above or from here

How to add Tokeninput to your database

Tokeninput consists of one js file and three css files. These can be added to your database by dragging and dropping them into the WebContents folder of the database (available through the Package Explorer)

Token2

The js files can be added to our database (along with the jQuery library) using the following XML markup in the XPage source

	<xp:this.resources>
		<xp:script src="js/jquery.tokeninput.js" clientSide="true"></xp:script>
		<xp:script src="js/jquery-1.7.1.min.js" clientSide="true"></xp:script>
		<xp:styleSheet href="css/Tokeninput/token-input.css"></xp:styleSheet>
		<xp:styleSheet href="css/Tokeninput/token-input-facebook.css"></xp:styleSheet>
		<xp:styleSheet href="css/Tokeninput/token-input-mac.css"></xp:styleSheet>
	</xp:this.resources>

Basic inner workings of Tokeninput

Tokeninput searches a given JSON string for a specified search parameter. As with many jQuery plugins there are set default which can be over-riden with user provided parmeters. By default the expected string format is:

	{{"id": "12345", "name": "Marky Roden"}]

and the code to add this to your XPage is as simple as the following where inputText3 is our text field and .tokenInput is out plugin instantiation:

	<xp:inputText value="#{sample.Custom}" id="inputText3">
	</xp:inputText>

	<xp:scriptBlock id="scriptBlock4">
		<xp:this.value>
		<![CDATA[
			$(document).ready(function() {
				$("#[id$=inputText3]").tokenInput("xTokeninput.xsp/contactjson", {
				  propertyToSearch: "email"
				});
			});
		]]>
		</xp:this.value>
	</xp:scriptBlock>

As we will see later, “xTokeninput.xsp/contactjson” is a REST service we are going to create on the page and propertyToSearch: “email” is the JSON string property we want to search

Example selecting from a REST service
Example selecting from a REST service

Changing the theme
Using the “theme” paramater we are able to determine the style in which the selected “tokens” are displayed to the user. Adding the facebook theme which comes with the plugin we change the way the results are displayed (not the results themselves).

			$(document).ready(function() {
				$("#[id$=inputText3]").tokenInput("xTokeninput.xsp/contactjson", {
				  propertyToSearch: "email",
				  theme: facebook
				});
			});
Example REST data with facebook theme
Example REST data with facebook theme

Data sources for Tokeninput

As you can see from the provided example we are able to GET (or POST) the JSON data from many sources, I have provided just a few examples, but really the options are only bounded by your needs and imagination.

A boring old Notes Agent

Before I get any grief for not using an XAgent the reason I use this as my first example is that it is probably the easiest to understand and will demonstrate how Tokeninput can generate and manage the search results itself (using it’s own internal ajax calls).
By default if you provide a URL (not a JSON object) to Tokeninput it will assume that is the URL to search and it will add “q=” to the end of the provide URL string, and look for the results. This really is as simple as it gets for Tokeninput, not a single parameter, all assumed default values. custom1 is the name of our text field and peopleLookup is our notes agent.

		$(document).ready(function() {
			$("#[id$=custom1]").tokenInput("peopleLookup?openagent?open&", {
			})
	    });

The peopleLookup agent provides a JSON string based on document fields values in a view, a snippet is below and the original is provided in the sample database

sQuery = StrRight(doc.Query_String_Decoded(0), "&q=")
Set docNAB = viewNAB.getdocumentbykey(sQuery)
i=0
Print {content-type: text/json}
	Print "["
	'loop for 10 results
	Do While Not docNAB Is Nothing And i<10

		'Because this ia JSON string each entry needs to have a comma after it except the first and last
		'so we add a comma because we know we are adding another value here
		If i>0 Then
			Print {, }
		End If
			Print |{"id":"|+docNAB.id(0)+|","name":"|+docNAB.FirstName(0)+| |+docNab.MiddleInitial(0)+| |+docNAB.LastName(0)+|"}|
			Set docNAB=viewNAB.getnextdocument(docNAB)
		i=i+1
	Loop
	Print "]"

As you can see from the picture below the JSON string maps directly to the Tokeninput displayed user options. The URL this JSON came from was http://demo.xomino.com/xomino/jQinX.nsf/peopleLookup?openagent?open&q=b

JSON data shown in firebug along with results
JSON data shown in firebug along with results

XPage REST Data Source

Using the fantastic OpenNTF extension library (and included out of the box in 8.5.3 UP1) we are able to add a REST service to the XPages. What does that mean? Well what the REST service allows us to do at its most basic is it allows us to create a JSON object from a notesview in a format that makes sense. Unlike the readviewentries option (discussed later in the article) the format created is easy to parse, easy to read and most importantly for Tokeninput, flat. If you want to take a look at the feed you will notice that it within the url of the example page (xTokeninput.xsp)

The code for the REST service is shown below:

	<xe:restService id="restService1" pathInfo="contactjson">
		<xe:this.service>
			<xe:viewJsonService viewName="AllContacts2" var="entry"
				contentType="text/plain" count="10">
				<xe:this.columns>
					<xe:restViewColumn name="id" columnName="id">
					</xe:restViewColumn>
					<xe:restViewColumn name="email"
						columnName="email">
					</xe:restViewColumn>
				</xe:this.columns>
				<xe:this.keys><![CDATA[#{javascript:return facesContext.getExternalContext().getRequest().getParameter("q")}]]></xe:this.keys>
			</xe:viewJsonService>
		</xe:this.service>
	</xe:restService>

and this is how it breaks down…..

  • viewName=”AllContacts2″ specifies the view which we want to pull the data from
  • var = Entry gives us access to the viewentry for further manipulation (see OpenNTF sample database for examples)
  • name = “name” is the name of the JSON Object [{“name”: “marky@whatever.com”}]
  • columnName = “email” is the programmatic name of the view column which is the object value [{“name”: “marky@whatever.com”}]
  • count=10 determines the number of results returned.

For each column we want provided in the feed we add a restViewColumn and give it the matching columnName and the JSON object value to display as (name=”id”)

The View Column Programmatic name must be used not the column Header
The View Column Programmatic name must be used not the column Header
	<xe:restViewColumn name="id" columnName="id"></xe:restViewColumn>

We can provide a “key” to the REST service which acts likes a getDocumentByKey.

	<xe:this.keys><![CDATA[#{javascript:return facesContext.getExternalContext().getRequest().getParameter("q")}]]></xe:this.keys>

The service is called every time the user enters a new key value “ma” would create the URL request to

http://demo.xomino.com/xomino/jQinX.nsf/xTokeninput.xsp/contactjson?q=ma

The key then extract the “ma” from the Query_String and provides the first document matching “ma” and then 10 documents after that. These become the displayed results from Tokeninput. A picture of the results was shown above in the initial discussion.

Standard Domino ?ReadViewEntries&OutputFormat=JSON

Since version 8 (officially) we have been able to get view data in a JSON format using the following type of URL
blah.nsf/viewname?ReadViewEntries&StarkKey=x&OutputFormat=JSON. But unfortunately the domino creation wizards at IBM decided to make the output format look more like a notesview than anything which works in Javascript (kinda the point of JSON). It ugly. So we have to work to get it into the format we need for Tokeninput. Click here to see an example of the JSON format and then you’ll have to take it to jsBeautifier.org to see the indentation. The major headache is that “@” values are properties and not objects values which makes for confusing code.

I have working proof of concept for this but I do not like the solution so it is really hard for me to write about it. With so many “better” ways to get the data from a notes view this is not something I recommend. If anyone is interested please contact me directly (@MarkyRoden) and we can discuss.

Twitter Data Source

The specification for the publically available twitter API is available from and the simplified format looks like this (looking at the feed for @MarkyRoden)

Click Here for the Feed link

[{
    "created_at": "Sat Apr 28 15:42:51 +0000 2012",
    "id": 196263215288684545,
    "id_str": "196263215288684545",
    "text": "@byrd1089 jQuery Tokeninput - hopefully will figure out how to parse readviewentires to the necessary format :)",
    "source": "\u003ca href=\"http:\/\/www.tweetdeck.com\" rel=\"nofollow\"\u003eTweetDeck\u003c\/a\u003e",
    "truncated": false,
    "in_reply_to_status_id": 196254762453303296,
    "in_reply_to_status_id_str": "196254762453303296",
    "in_reply_to_user_id": 92247920,
    "in_reply_to_user_id_str": "92247920",
    "in_reply_to_screen_name": "byrd1089",
    "user": {
        "id": 463110112,
        "id_str": "463110112",
        "name": "Mark Roden",
        "screen_name": "MarkyRoden",
        "location": "Reston, VA USA",

The source code for my example twitter pull is below. What I am doing is loading the data via “screen_name” value (e.g. edbrill or openntf) for the last 100 entries and returning the 10 entries matching the search parameter.  The parameters do the following thing:

  • propertyToSearch: the field to be searched “text”
  • queryParam: the Object name to query from the API “screen_name”
  • tokenFormatted: the HTML describing the way the results are displayed to the user
  • results formatted: the HTML describing the way the select result is stored
  • onResult: a callback function (in this case correcting a bug in Tokeninput replacing ( and ) in the return string)
			$(document).ready(function() {
				$("#[id$=inputText6]").tokenInput("https://api.twitter.com/1/statuses/user_timeline.json?include_entities=true&include_rts=true&count=10", {
					propertyToSearch: "text",
					queryParam: "screen_name",
					resultsFormatter: function(item){ return "<li>" + "<img src='" + item.user.profile_image_url + "' title='" + item.user.screen_name + "' height='40px' width='40px' />" + "<div style='display: inline-block; padding-left: 10px; width:80%'><div class='full_name'>" + item.text  + "</div></div></li>" },
					tokenFormatter: function(item) { return "<li>" + "<img src='" + item.user.profile_image_url + "' title='" + item.user.screen_name + "' height='40px' width='40px' />" + "<div style='display: inline-block; padding-left: 10px; width:80%'><div class='full_name'>" + item.text  + "</div></div></li>" },
					onResult: function (results) {
      					jQuery.each(results, function (index, value) {
				        	value.text = value.text.replace(/\(/g, '(').replace(/\)/g, ')');
						});
				      return results;
				    }
			    });
		    });

And this to me with where Tokeninput takes the typeahead autocomplete to the next level…..

Example Twitter feed with formatted results
Example Twitter feed with formatted results

so what happens when this is submitted? If you remember the default submitted value in the Tokeninput is “id”….and If you look at the feed there is an id tag for every tweet…..

([{
    "created_at": "Sat Apr 28 15:42:51 +0000 2012",
    "id": 196263215288684545,
    "id_str": "196263215288684545",
    "text": "@byrd1089 jQuery Tokeninput - hopefully will figure out how to parse readviewentires to the necessary format :)",
    "source": "\u003ca href=\"http:\/\/www.tweetdeck.com\" rel=\"nofollow\"\u003eTweetDeck\u003c\/a\u003e",
    "truncated": false,

and when we submit the value back to the database we get a multivalue field containing all the tweet ids. If you want to change the field stored use the “tokenValue” parameter. The Tokeninput field is mapped to the “Custom” field in the XPage through the data source.

Showing the submitted tweet ids stored in the notesdocument
Showing the submitted tweet ids stored in the notesdocument

Scratching the surface

The possibilities with Tokeninput are only limited by the developers imagination and the data source available. I have demonstrated how we can create our own JSON Feed using a REST feed, how to create a simple JSON feed using a notesagent, how to parse an existing notes view readviewentries JSON feed and how to parse an external feed (twitter) and display some fantastic result results while maintaining data control.

In my opinion (from a user experience) this trumps the XPages typeAhead field capability hands down . This also trumps the XPages typeAhead from a developer perspective as well because it will take data from sources other than an internal notes view and will return multiple values which the current type ahead will not.

Demonstration site

This article is published in conjunction with the original post and  demonstration site

And then what?

To be submitted to OpenNTF – I am working on a custom control to add a dynamic Tokeninput field to any XPage. It will provide a direct alternative to the dojo typeAhead and provide greater flexibility for users.

It’s just going to be ready for this week’s article 🙂

Work in progress - custom control for adding tokenizer to an XPage using REST
Work in progress - custom control for adding tokenizer to an XPage using REST