Paginating Backbone.js Requests and Collections - Developing Backbone.js Applications (2013)

Developing Backbone.js Applications (2013)

Chapter 10. Paginating Backbone.js Requests and Collections

Pagination is a ubiquitous problem we often find ourselves needing to solve on the Web—perhaps most predominantly when we’re working with service APIs and JavaScript-heavy clients that consume them. It’s also a problem that is often underrefined because most of us consider pagination relatively easy to get right. This isn’t, however, always the case, as pagination tends to get trickier than it initially seems.

Before we dive into solutions for paginating data for your Backbone applications, let’s define exactly what we consider pagination to be.

Pagination is a control system allowing users to browse through pages of search results or any type of content that is continued. Search results are the canonical example, but pagination today is found on news sites, blogs, and discussion boards, often in the form of Previous and Next links. More complete pagination systems offer granular control of the specific pages you can navigate to, giving users more power to find what they are looking for.

It isn’t a problem limited to pages requiring some visual controls for pagination either—sites like Facebook, Pinterest, and Twitter have demonstrated that there are many contexts where infinite paging is also useful. Infinite paging is, of course, when we prefetch (or appear to prefetch) content from a subsequent page and add it directly to the user’s current page, making the experience feel infinite.

Pagination is very context-specific and depends on the content being displayed. In the Google search results, pagination is important because Google wants to offer you the most relevant set of results in the first one or two pages. After that, you might be a little more selective (or random) with the page you choose to navigate to. This differs from cases where you’ll want to cycle through consecutive pages (for example, for a news article or blog post).

Pagination is almost certainly content and context-specific, but as Faruk Ates has previously pointed out, the principles of good pagination apply no matter what the content or context is. As with everything extensible when it comes to Backbone, you can write your own pagination to address many of these content-specific types of pagination problems. That said, you’ll probably spend quite a bit of time on this, and sometimes you want to use a tried-and-true solution that just works.

On this topic, we’re going to go through a set of pagination components that I and a group of contributors wrote for Backbone.js, which should hopefully come in handy if you’re working on applications that need to page Backbone collections. These components are part of an extension called Backbone.Paginator.

Backbone.Paginator

When working with data on the client side, we are most likely to run into three types of pagination:

Requests to a service layer (API)

For example, query for results containing the term Paul—if 5,000 results are available, display only 20 results per page (leaving us with 250 possible result pages that can be navigated to).

This problem actually has quite a great deal more to it, such as maintaining persistence of other URL parameters (such as sort, query, order) that can change based on a user’s search configuration in a UI. You also have to think of a clean way to hook up views to this pagination so you can easily navigate between pages (for example, First, Last, Next, Previous, 1, 2, 3), manage the number of results displayed per page, and so on.

Further client-side pagination of data returned

For example, we’ve been returned a JSON response containing 100 results. Rather than displaying all 100 to the user, we display only 20 of these results within a navigable UI in the browser.

Similar to the request problem, client pagination has its own challenges, like navigation once again (Next, Previous, 1, 2, 3), sorting, order, switching the number of results to display per page, and so on.

Infinite results

With services such as Facebook, the concept of numeric pagination is instead replaced with a Load More or View More button. Triggering this normally fetches the next page of N results, but rather than replacing entirely the previous set of results loaded, we simply append to them instead.

A request pager, which simply appends results in a view rather than replacing on each new fetch, is effectively an infinite pager.

Let’s now take a look at exactly what we’re getting out of the box.

Backbone.Paginator, shown in Figure 10-1, is a set of opinionated components for paginating collections of data using Backbone.js. It aims to provide both solutions for assisting with pagination of requests to a server (such as an API) as well as pagination of single loads of data, where we may wish to further paginate a collection of N results into M pages within a view.

Backbone.Paginator demonstrating how to visually style the components provided by the project

Figure 10-1. Backbone.Paginator demonstrating how to visually style the components provided by the project

Backbone.Paginator supports two main pagination components:

Backbone.Paginator.requestPager

For pagination of requests between a client and a server-side API

Backbone.Paginator.clientPager

For pagination of data returned from a server that you would like to further paginate within the UI (for example, 60 results are returned, paginate into three pages of 20)

Live Examples

If you would like to look at examples built using the components included in the project, links to official demos are included here and use the Netflix API so that you can see them working with an actual data source:

§ Backbone.Paginator.requestPager()

§ Backbone.Paginator.clientPager()

§ Infinite pagination (Backbone.Paginator.requestPager())

§ Diacritic plug-in

Paginator.requestPager

In this section we’re going to walk through using the requestPager (shown in Figure 10-2). You would use this component when working with a service API that itself supports pagination. This component allows users to control the pagination settings for requests to this API (for example, navigate to the next, previous, N pages) via the client side.

The idea is that pagination, searching, and filtering of data can all be done from your Backbone application without the need for a page reload.

Using the requestPager component to request paginated results from the Netflix API

Figure 10-2. Using the requestPager component to request paginated results from the Netflix API

1. Create a new Paginated collection.

First, we define a new Paginated collection using Backbone.Paginator.requestPager() as follows:

var PaginatedCollection = Backbone.Paginator.requestPager.extend({

2. Set the model for the collection as normal.

Within our collection, we then (as normal) specify the model to be used with this collection followed by the URL (or base URL) for the service providing our data (such as the Netflix API).

model: model,

3. Configure the base URL and the type of request.

We need to set a base URL. The type of the request is GET by default, and the dataType is jsonp in order to enable cross-domain requests.

paginator_core: {

// the type of the request (GET by default)

type: 'GET',

// the type of reply (jsonp by default)

dataType: 'jsonp',

// the URL (or base URL) for the service

// if you want to have a more dynamic URL, you can make this

// a function that returns a string

url: 'http://odata.netflix.com/Catalog/People(49446)/TitlesActedIn?'

},

WARNING

If you use dataType not jsonp, please remove the callback custom parameter inside the server_api configuration.

4. Configure how the library will show the results.

We need to tell the library how many items per page we would like to display, what the current page is, what the range of pages should be, and so on.

paginator_ui: {

// the lowest page index your API allows to be accessed

firstPage: 0,

// which page should the paginator start from

// (also, the actual page the paginator is on)

currentPage: 0,

// how many items per page should be shown

perPage: 3,

// a default number of total pages to query in case the API or

// service you are using does not support providing the total

// number of pages for us.

// 10 as a default in case your service doesn't return the total

totalPages: 10

},

5. Configure the parameters we want to send to the server.

The base URL won’t be enough for most cases, so you can pass more parameters to the server. Note how you can use functions instead of hardcoded values, and you can also refer to the values you specified in paginator_ui.

server_api: {

// the query field in the request

'$filter': '',

// number of items to return per request/page

'$top': function() { return this.perPage },

// how many results the request should skip ahead to

// customize as needed. For the Netflix API, skipping ahead based on

// page * number of results per page was necessary.

'$skip': function() { return this.currentPage * this.perPage },

// field to sort by

'$orderby': 'ReleaseYear',

// what format would you like to request results in?

'$format': 'json',

// custom parameters

'$inlinecount': 'allpages',

'$callback': 'callback'

},

WARNING

If you use $callback, please ensure that you did use jsonp as a dataType inside your paginator_core configuration.

6. Finally, configure Collection.parse(), and we’re done.

The last thing we need to do is configure our collection’s parse() method. We want to ensure we’re returning the correct part of our JSON response containing the data our collection will be populated with, which in the following is response.d.results (for the Netflix API).

parse: function (response) {

// Be sure to change this based on how your results

// are structured (e.g., d.results is Netflix-specific)

var tags = response.d.results;

//Normally this.totalPages would equal response.d.__count

//but as this particular NetFlix request only returns a

//total count of items for the search, we divide.

this.totalPages = Math.ceil(response.d.__count / this.perPage);

return tags;

}

});

});

You might also notice that we’re setting this.totalPages to the total page count returned by the API. This allows us to define the maximum number of (result) pages available for the current/last request so that we can clearly display this in the UI. It also allows us to influence whether clicking, say, a Next button should proceed with a request or not.

Convenience Methods

For your convenience, the following methods are made available for use in your views to interact with the requestPager:

Collection.goTo( n, options )

Go to a specific page

Collection.nextPage( options )

Go to the next page

Collection.prevPage( options )

Go to the previous page

Collection.howManyPer( n )

Set the number of items to display per page

The requestPager collection’s methods .goTo(), .nextPage(), and .prevPage() are all extensions of the original Backbone Collection.fetch() methods. As such, they all can take the same option object as a parameter.

This option object can use success and error parameters to pass a function to be executed after server answers.

Collection.goTo(n, {

success: function( collection, response ) {

// called if server request success

},

error: function( collection, response ) {

// called if server request fail

}

});

To manage callback, you could also use the jqXHR returned by these methods.

Collection

.requestNextPage()

.done(function( data, textStatus, jqXHR ) {

// called if server request success

})

.fail(function( data, textStatus, jqXHR ) {

// called if server request fail

})

.always(function( data, textStatus, jqXHR ) {

// do something after server request is complete

});

});

If you’d like to add the incoming models to the current collection, instead of replacing the collection’s contents, pass {update: true, remove: false} as options to these methods.

Collection.prevPage({ update: true, remove: false });

Paginator.clientPager

The clientPager (Figure 10-3) is used to further paginate data that has already been returned by the service API. Say you’ve requested 100 results from the service and wish to split this into five pages of paginated results, each containing 20 results at a client level—the clientPagermakes it trivial to do this.

Using the clientPager component to further paginate results returned from the Netflix API

Figure 10-3. Using the clientPager component to further paginate results returned from the Netflix API

Use the clientPager when you prefer to get results in a single load and thus avoid making additional network requests each time your users want to fetch the next page of items. As the results have all already been requested, it’s just a matter of switching between the ranges of data actually presented to the user.

1. Create a new paginated collection with a model and URL.

As with requestPager, let’s first create a new paginated Backbone.Paginator.clientPager collection, with a model:

var PaginatedCollection = Backbone.Paginator.clientPager.extend({

model: model,

2. Configure the base URL and the type of request.

We need to set a base URL. The type of the request is GET by default, and the dataType is jsonp in order to enable cross-domain requests.

paginator_core: {

// the type of the request (GET by default)

type: 'GET',

// the type of reply (jsonp by default)

dataType: 'jsonp',

// the URL (or base URL) for the service

url: 'http://odata.netflix.com/v2/Catalog/Titles?&'

},

3. Configure how the library will show the results.

We need to tell the library how many items per page we would like to display, what the current page is, what the range of pages should be, and so on.

paginator_ui: {

// the lowest page index your API allows to be accessed

firstPage: 1,

// which page should the paginator start from

// (also, the actual page the paginator is on)

currentPage: 1,

// how many items per page should be shown

perPage: 3,

// a default number of total pages to query in case the API or

// service you are using does not support providing the total

// number of pages for us.

// 10 as a default in case your service doesn't return the total

totalPages: 10,

// The total number of pages to be shown as a pagination

// list is calculated by (pagesInRange * 2) + 1.

pagesInRange: 4

},

4. Configure the parameters we want to send to the server.

The base URL alone won’t be enough for most cases, so you can pass more parameters to the server. Note how you can use functions instead of hardcoded values, and you can also refer to the values you specified in paginator_ui.

server_api: {

// the query field in the request

'$filter': 'substringof(\'america\',Name)',

// number of items to return per request/page

'$top': function() { return this.perPage },

// how many results the request should skip ahead to

// customize as needed. For the Netflix API, skipping ahead based on

// page * number of results per page was necessary.

'$skip': function() { return this.currentPage * this.perPage },

// field to sort by

'$orderby': 'ReleaseYear',

// what format would you like to request results in?

'$format': 'json',

// custom parameters

'$inlinecount': 'allpages',

'$callback': 'callback'

},

5. Finally, configure Collection.parse(), and we’re done.

And finally we have our parse() method, which in this case isn’t concerned with the total number of result pages available on the server, as we have our own total count of pages for the paginated data in the UI.

parse: function (response) {

var tags = response.d.results;

return tags;

}

});

Convenience Methods

As mentioned, your views can hook into a number of convenience methods to navigate around UI-paginated data. For clientPager, these include:

Collection.goTo( n, options )

Go to a specific page.

Collection.prevPage( options )

Go to the previous page.

Collection.nextPage( options )

Go to the next page.

Collection.howManyPer( n )

Set how many items to display per page.

Collection.setSort( sortBy, sortDirection )

Update sort on the current view. Sorting will automatically detect if you’re trying to sort numbers (even if they’re stored as strings) and will do the right thing.

Collection.setFilter( filterFields, filterWords )

Filter the current view. Filtering supports multiple words without any specific order, so you’ll basically get a full-text search ability. Also, you can pass it only one field from the model, or you can pass an array with fields and all of them will get filtered. The last option is to pass it an object containing a comparison method and rules. Currently, only the Levenshtein method is available. The Levenshtein distance is the difference between two strings and is effectively the minimum number of changes required to change one word into another.

The goTo(), prevPage(), and nextPage() functions do not require the options param since they will be executed synchronously. However, when specified, the success callback will be invoked before the function returns. For example:

nextPage(); // this works just fine!

nextPage({success: function() { }}); // this will call the success function

The options param exists to preserve (some) interface unification between the requestPaginator and clientPaginator so that they may be used interchangeably in your Backbone.Views.

this.collection.setFilter(

{'Name': {cmp_method: 'levenshtein', max_distance: 7}}

, "American P" // Note the switched 'r' and 'e', and the 'P' from 'Pie'

);

Also note that the Levenshtein plug-in should be loaded and enabled via the useLevenshteinPlugin variable. Last but not less important: performing Levenshtein comparison returns the distance between two strings. It won’t let you search lengthy text. The distance between two strings means the number of characters that should be added, removed, or moved to the left or to the right so the strings get equal. That means that comparing “Something” in “This is a test that could show something” will return 32, which is bigger than comparing “Something” and “ABCDEFG (9).” Use Levenshtein only for short texts (titles, names, and so on).

Collection.doFakeFilter( filterFields, filterWords )

Returns the models count after fake-applying a call to Collection.setFilter.

Collection.setFieldFilter( rules )

Filter each value of each model according to rules that you pass as an argument. Say you have a collection of books with release year and author. You can filter only the books that were released between 1999 and 2003. And then you can add another rule that will filter those books only to authors whose name starts with A. Possible rules: function, required, min, max, range, minLength, maxLength, rangeLength, oneOf, equalTo, containsAllOf, pattern. Passing this an empty rules set will remove any FieldFilter rules applied.

my_collection.setFieldFilter([

{field: 'release_year', type: 'range', value:

{min: '1999', max: '2003'}},

{field: 'author', type: 'pattern', value: new RegExp('A*', 'igm')}

]);

//Rules:

//

//var my_var = 'green';

//

//{field: 'color', type: 'equalTo', value: my_var}

//{field: 'color', type: 'function', value: function(field_value){

return field_value == my_var; } }

//{field: 'color', type: 'required'}

//{field: 'number_of_colors', type: 'min', value: '2'}

//{field: 'number_of_colors', type: 'max', value: '4'}

//{field: 'number_of_colors', type: 'range', value: {min: '2', max: '4'} }

//{field: 'color_name', type: 'minLength', value: '4'}

//{field: 'color_name', type: 'maxLength', value: '6'}

//{field: 'color_name', type: 'rangeLength', value: {min: '4', max: '6'}}

//{field: 'color_name', type: 'oneOf', value: ['green', 'yellow']}

//{field: 'color_name', type: 'pattern', value: new RegExp('gre*', 'ig')}

//{field: 'color_name', type: 'containsAllOf', value:

['green', 'yellow', 'blue']}

Collection.doFakeFieldFilter( rules )

Returns the models count after fake-applying a call to Collection.setFieldFilter.

Implementation Notes

You can use some variables in your view to represent the actual state of the paginator.

totalUnfilteredRecords

Contains the number of records, including all records filtered in any way (available only in clientPager).

totalRecords

Contains the number of records.

currentPage

The actual page where the paginator is located.

perPage

The number of records the paginator will show per page.

totalPages

The number of total pages.

startRecord

The position of the first record shown in the current page—for example, 41 to 50 from 2,000 records (available only in clientPager).

endRecord

The position of the last record shown in the current page—for example, 41 to 50 from 2,000 records (available only in clientPager).

pagesInRange

The number of pages to be drawn on each side of the current page. So, if pagesInRange is 3 and currentPage is 13, you will get the numbers 10, 11, 12, 13 (selected), 14, 15, 16.

<!-- sample template for pagination UI -->

<script type="text/html" id="tmpServerPagination">

<div class="row-fluid">

<div class="pagination span8">

<ul>

<% _.each (pageSet, function (p) { %>

<% if (currentPage == p) { %>

<li class="active"><span><%= p %></span></li>

<% } else { %>

<li><a href="#" class="page"><%= p %></a></li>

<% } %>

<% }); %>

</ul>

</div>

<div class="pagination span4">

<ul>

<% if (currentPage > firstPage) { %>

<li><a href="#" class="serverprevious">Previous</a></li>

<% }else{ %>

<li><span>Previous</span></li>

<% }%>

<% if (currentPage < totalPages) { %>

<li><a href="#" class="servernext">Next</a></li>

<% } else { %>

<li><span>Next</span></li>

<% } %>

<% if (firstPage != currentPage) { %>

<li><a href="#" class="serverfirst">First</a></li>

<% } else { %>

<li><span>First</span></li>

<% } %>

<% if (totalPages != currentPage) { %>

<li><a href="#" class="serverlast">Last</a></li>

<% } else { %>

<li><span>Last</span></li>

<% } %>

</ul>

</div>

</div>

<span class="cell serverhowmany"> Show <a href="#"

class="selected">18</a> | <a href="#" class="">9</a> |

<a href="#" class="">12</a> per page

</span>

<span class="divider">/</span>

<span class="cell first records">

Page: <span class="label"><%= currentPage %></span> of

<span class="label"><%= totalPages %></span> shown

</span>

</script>

Plug-ins

Diacritic.js is a plug-in for Backbone.Paginator that replaces diacritic characters (´, ˝, ˚, ~, and so on) with characters that match them most closely, as shown in Figure 10-4. This is particularly useful for filtering.

The Diacritics plug-in being used to correctly render special characters with the clientPager

Figure 10-4. The Diacritics plug-in being used to correctly render special characters with the clientPager

To enable the plug-in, set this.useDiacriticsPlugin to true, as shown in this example:

Paginator.clientPager = Backbone.Collection.extend({

// Default values used when sorting and/or filtering.

initialize: function(){

this.useDiacriticsPlugin = true; // use diacritics plug-in if available

...

Bootstrapping

By default, both the clientPager and requestPager will make an initial request to the server in order to populate their internal paging data. To avoid this additional request, you may find it beneficial to bootstrap your Backbone.Paginator instance from data that already exists in the DOM, as shown here in Backbone.Paginator.clientPager.

// Extend the Backbone.Paginator.clientPager with your own configuration options

var MyClientPager = Backbone.Paginator.clientPager.extend({paginator_ui: {}});

// Create an instance of your class and populate with the models of your

// entire collection

var aClientPager = new MyClientPager([{id: 1, title: 'foo'},

{id: 2, title: 'bar'}]);

// Invoke the bootstrap function

aClientPager.bootstrap();

NOTE

If you intend to bootstrap a clientPager, there is no need to specify a paginator_core object in your configuration (since you should have already populated the clientPager with the entirety of its necessary data), as shown here in Backbone.Paginator.requestPager.

// Extend the Backbone.Paginator.requestPager with your own configuration options

var MyRequestPager = Backbone.Paginator.requestPager.extend({paginator_ui: {}});

// Create an instance of your class with the first page of data

var aRequestPager = new MyRequestPager([{id: 1, title: 'foo'},

{id: 2, title: 'bar'}]);

// Invoke the bootstrap function and configure requestPager with 'totalRecords'

aRequestPager.bootstrap({totalRecords: 50});

NOTE

Both the clientPager and requestPager bootstrap function will accept an options parameter that will be extended by your Backbone.Paginator instance. However, the totalRecords property will be set implicitly by the clientPager.

For more on Backbone bootstrapping, see Rico Sta Cruz’s website.

Styling

You’re free, of course, to customize the overall look and feel of the paginators as much as you wish. By default, all sample applications make use of the Twitter Bootstrap for styling links, buttons, and drop-downs.

CSS classes are available to style record counts, filters, sorting, and more, as shown in Figure 10-5.

Inspecting the Paginator using the Chrome DevTool console provides insights into some of the classes’ support for styling

Figure 10-5. Inspecting the Paginator using the Chrome DevTool console provides insights into some of the classes’ support for styling

Classes are also available for styling more granular elements like page counts within breadcrumb > pages (for example, .page, .page selected), as shown in Figure 10-6.

A demonstration of how pagination can be styled using a Twitter Bootstrap breadcrumb

Figure 10-6. A demonstration of how pagination can be styled using a Twitter Bootstrap breadcrumb

There’s a tremendous amount of flexibility available for styling, and as you’re in control of templating too, your paginators can be made to look as visually simple or complex as needed.

Summary

Although it’s certainly possible to write your own custom pagination classes to work with Backbone collections, Backbone.Paginator tries to take care of much of this for you.

It’s highly configurable, preventing you from having to write your own paging when working with collections of data sourced from your database or API. Use the plug-in to help tame large lists of data into more manageable, easily navigatable, paginated lists.

Additionally, if you have any questions about Backbone.Paginator (or would like to help improve it), feel free to post to the project issues list.