Couchbase Essentials (2015)

Chapter 7. Creating a To-do App with Couchbase

In this chapter, we'll put together everything you learned so far. In recent years, the to-do app has replaced the blog as the canonical first app when learning a new platform. While a to-do app seems simple on the surface, it is complex enough to demonstrate most of the core features of a framework.

A to-do app built on Couchbase is well suited to demonstrate both the key/value and document features of Couchbase. With some minor feature additions to a typical to-do app, we'll be able to make use of some of the advanced view features available for Couchbase developers.

Throughout this chapter, we'll focus more on general design considerations rather than specific SDK or language constructs. As was the case in the previous chapters, we'll explore multiple SDKs as we build our to-do app. While the basic language constructs may vary from SDK to SDK, the broad strokes approach will not vary.

A simple to-do schema

You've learned in previous chapters that schema design in the world of schema-less NoSQL databases tends to derive from the logical or object design of the application layer. As such, we'll start our application development efforts by considering the design of the classes we'll use in our application.

In the simplest case, a to-do app is nothing more than a checklist. To start modeling our schema, we'll limit the design to two properties of a checklist, namely a description and a checkbox. This design is shown in the following C# class:

public class Task

{

public string Description { get; set; }

public bool IsComplete { get; set; }

}

In this class, the Description property describes the task to be done. The Boolean property IsComplete simply checks whether the task has been completed. The corresponding JSON document stored in Couchbase mirrors this class:

{

"description": "Pick up the almond milk",

"isComplete": false

}

As we build our application, we'll add more features and develop our schema further. For now however, we'll start building the application to support our simple task list.

Working with SDKs

Again, it's not feasible within the scope of a single chapter to implement an application with a single framework that would satisfy all readers. Even a cross-platform language such as Python would require a rather lengthy exploration into setting up a development environment and exploring a web framework and its components.

Java and .NET are quite popular platforms, but require a fair bit of tooling support to get these platforms up and running. Focusing exclusively on one of these platforms would almost certainly alienate a significant number of readers. And, of course, there are differences across Windows, Linux, and Mac OS X.

Instead, we'll focus on the general principles and patterns of Couchbase SDK development. We'll explore constructs that will be broadly applicable to developing a Couchbase application, regardless of your development environment.

Also, we won't dig into any particular web framework but will discuss general web development patterns. Chances are that if you're a web developer and you're working with a NoSQL database, you're likely using a framework that supports capabilities like MVC(short for model-view-controller).

A brief overview of MVC

For the purpose of this chapter, all you'll need to know of MVC or a similarly-patterned web framework is that when you navigate to a URI such as http://localhost/tasks/list, you will have a corresponding server-side method that handles the request to the listaction. Similarly, a request to http://localhost/tasks/create would have a corresponding create action. Actions are simply methods invoked on the server that handle an HTTP request and return an HTTP response.

For example, using the popular ASP.NET MVC framework, if you wanted to show a form to create a task when a user navigated to http://localhost/tasks/create, you would create a controller named tasks and a method (or action) named create:

public class TasksController : Controller

{

[HttpGet]

public ActionResult Create()

{

return View();

}

As is common with MVC, you create a Controller class where the name of the controller reflects some portion of the requested URI path (tasks in this case). Within that controller, you define a method, and that method is also reflected in the path (create in this case). The previous ASP.NET MVC snippet shows a create action that will handle only HttpGet requests.

Similarly, to handle the postback data from the HTML form to the server, you create an action to handle the form submission. In ASP.NET MVC, this is done by placing an HttpPost attribute on a method matching the name of the action:

[HttpPost]

public ActionResult Create(FormCollection form)

{

//do something with the form

return RedirectToAction("List");

}

Most MVC frameworks follow a similar convention. MVC frameworks such as Ruby's Rails vary slightly in how they handle the different HTTP verbs presented to an action:

class TasksController<ApplicationController

def new

end

def create

#do something

redirect_to :action => "Index"

end

In this Rails snippet, we see a similar convention as used by MVC, the primary difference being that the ASP.NET MVC uses attributes to distinguish between GET and POST actions. Other frameworks have a single method that checks the verb to decide how to perform operations.

A variation of popular MVC frameworks is a so-called micro framework. Generally speaking, you could think of a web micro framework as an MVC framework without the "C" (that is, the controller). With such frameworks, you'll typically define the path handled by an action, without a controller involved.

A popular micro framework in the Python world is Flask. With Flask, you set up a series of routes and instruct Flask on how to dispatch requests to the appropriate handlers. For example, to handle the simple rendering of a create view, the following flask snippet would be used:

@app.route("/tasks/create")

def create():

return render_view("create.html")

That same method can be expanded to handle the post back of data, as follows:

@app.route("/tasks/create")

def create():

if request.method == "POST":

#do something

return redirect(url_for("index"))

return render_view("create.html")

If you are already familiar with a web framework, the preceding samples should seem familiar. If you have not used a web framework, then you will see snippets like these throughout the remainder of this chapter. For our purpose, it's most important that you have a basic understanding of what these action methods are doing, rather than detailed knowledge of a particular framework.

Using SDK clients

In Chapter 2, Using Couchbase CRUD Operations, we explored the basics of obtaining SDK client libraries. Generally, this was achieved via your platform's package manager (for example, NuGet, Gems, or PIP). Assuming that you've obtained your platform's SDK, the first thing you'll need to understand is how to configure and instantiate that client.

Regardless of which SDK you are using, each SDK requires the same basic setup configuration—the location of a node in your cluster and the bucket with which you want to connect. The Python SDK demonstrates this process succinctly:

from couchbase import Couchbase

client = Couchbase.connect(host = "localhost", bucket = "beer-sample")

When a Couchbase SDK connects to a node in your cluster, it begins listening to a streaming (over HTTP) message from the server. The content sent through this stream provides the SDK with information on the topology of the cluster, such as how many nodes are active in the cluster and where keys should be sent to or requested from.

This handshake is relatively expensive, and therefore it is generally best practice not to create a client instance except when necessary. Within the scope of a web application, you'd want to have a single client handle all requests, typically by creating a static or shared instance of your client.

In the preceding Python snippet, the connect method is provided with limited details about the cluster. Also, in the Python snippet there are a few default values (such as ports) being used by the client. Similarly, the .NET 2.0 SDK may be configured with all defaults that connect to your localhost and default bucket:

private static Cluster _cluster = new Cluster();

var bucket = _cluster.OpenBucket();

The story is similar for other SDKs. You'll create a client by connecting to the cluster and then a bucket. If you're wondering which node in a cluster should be provided in the initial connection, the short answer is any. However, it is better to provide multiple nodes in case the node you specified undergoes failover.

The SDKs offer a means of providing multiple URIs via either a configuration file or parameters to connection methods. For example, in Java you could provide multiple URIs to the create factory method of the CouchbaseCluster class:

Cluster cluster = new CouchbaseCluster.create("192.168.0.1", "192.168.0.2");

In this example, if the first URI is not accessible, the SDK would then try to obtain information about cluster configuration from the second URI. How many nodes you should specify depends on your cluster, but generally, at least two and up to three or four nodes should be reasonable.

Creating a task

At this point, we've designed a very simple to-do schema where our tasks are simply checklist items. Regardless of which web framework you are using, you'll need some sort of HTML form to collect the description and isComplete properties of the new tasks:

<html>

<head>

<title>Create a Task</title>

</head>

<body>

<div>Description:

</div>

<div>Complete:

</div>

</div><button type="submit" value="Save" />

</form>

</body>

</html>

The preceding HTML form collects these two properties and submits them to a server-side action named create. As an example of how you can respond to this form post, consider the following Python Flask snippet:

@app.route("/tasks/create", methods=["GET","POST"])

def create():

if request.method == "POST":

task = { "description": request.form["description"],

"isComplete": request.form["isComplete"] }

key = uuid.uuid1().hex

doc = json.dumps(task)

client.set(key, doc)

return redirect(url_for("index"))

return render_view("create.html")

We can see the basic pattern of creating new documents with Couchbase in the preceding lines. These steps are similar to those you'd perform when working with a relational database, but there are a couple of differences.

In this example, the task is constructed as a Python dictionary instance. Alternatively, we could have used a class with properties matching the task fields. Because there is no obvious property of the task to use as a key, a UUID is generated and used as the key for the document. Finally, before saving the task to Couchbase, it is serialized to a JSON document using Python's JSON module.

These last two steps are the primary difference between Couchbase and other databases. Couchbase Server doesn't provide a means to generate keys, so we need to generate our own. Couchbase clients don't enforce JSON as a serialization format, so we need to take care of this ourselves.

Listing tasks

In the preceding snippet, after the task is created, a redirect to an index page is performed. This page is a list page used to view tasks. Building a list page requires finding all our tasks that will require a slight change to our model:

public class Task

{

public string Description { get; set; }

public bool IsComplete { get; set; }

public bool Type { get { return "task"; }

}

Recall our discussion from the previous chapters on the use of a type property on documents to provide a classification for related documents, much in the way a table does for relational databases. In our to-do application, to identify tasks, we'll add a type property (which is read-only). The property is set to the task string, which will ensure that all task documents are serialized with this type. With this addition, we're ready to write our list page, starting with a map function:

//view named "all" in a design doc "tasks"

function(doc, meta) {

if (doc.type == "task") {

emit(null, null);

}

Notice that this map function doesn't explicitly index any properties of task documents. Since we are indexing only documents marked as tasks, a query on this view will return only the documents we wish to list on our index page. The following C# example is intentionally verbose to illustrate a couple of points. Note that the .NET 1.3 SDK provides some helper methods to achieve similar 'margin-top:6.0pt;margin-right:29.0pt;margin-bottom: 6.0pt;margin-left:29.0pt;line-height:normal'>public ActionResult Index()

{

var view = client.GetView<Task>("all"", "tasks");

var model = GetTasksFromView(view);

return View(model);

}

private IEnumerable<Task>GetTasksFromView(IView view) {

foreach(var row in view)

{

var doc = client.Get<string>(row.ItemId);

yield return JsonConvert.DeserializeObject<Task>(doc);

}

The Index action starts by querying the all view in the tasks design document. Once the index of the results has been returned, the view is converted to an enumerable list of Task instances. In C#, yield return allows a function to be treated as an enumerable object, which means that the casting of a view row to a Task instance occurs only when the caller enumerates the results. In this case, the client is an ASP.NET MVC Razor view:

<table>

<thead>

<tr>

<th>Description</th>

<th>Complete</th>

</tr>

</thead>

@foreach(var item in Model)

{

<tr>

<td>@item.Description</td>

<td>@item.IsComplete</td>

</tr>

}

</table>

Regardless of which web framework you work with, the basic idea will be the same. You'll query a view, get the results, and pass those results to a view to be displayed. Once this list is complete, the next logical step is to allow the editing of tasks. This action will be similar to the create task we've worked on before.

To allow editing, we'll need to allow users to get to the edit page for a specific task. This requirement will necessitate adding a property to our class to map to the document's key. This change is important because we must be careful how to map our key, ensuring that JSON serializers don't include a store inside the document:

public class Task

{

[JsonIgnore]

public string Id { get; set; }

//other properties omitted

}

How you exclude the Id property from being serialized into the stored Couchbase document will of course vary by your language and its preferred serializer. In this snippet, an attribute is included on the Id property to instruct the JSON.NET serializer to ignore this property.

Although you're ignoring the Id property as the document is sent to Couchbase, it's important to remember that you'll want to set it to come out of the bucket. We'll see how to do this by modifying the preceding method that retrieves all task documents for listing:

private IEnumerable<Task>GetTasksFromView(IView view) {

foreach(var row in view)

{

var doc = client.Get<string>(row.ItemId);

//this line will populate all task properties,

//except for Id

var task = JsonConvert.DeserializeObject<Task>(doc);

//explicitly map the document's key to the

//Id property of the Task instance

task.Id = row.ItemId;

}

We'll also want to modify our list's view code so that we can create a link to the edit action we're about to build. In this case, we're simply wrapping the task's description column with a link to the edit page, with the item's id property passed as a parameter to the request. If you're using Flask with its default Jinja2 templating engine, the list would look like this:

{% for item in model %}

<tr>

<td><a href="/tasks/edit/{{item.id }}">{{ item.description }}</a>

<td>{{ item.is_complete }}</td>

</tr>

{% endfor %}

The edit method on the server will look a bit like the create method, except that it will modify the saved document on submission and return the unchanged saved document when the form is displayed:

@app.route("/tasks/edit/<key>", methods=["GET","POST"])

def edit(key):

saved_doc = client.get(key) #retrieve from the bucket

#deserializesaved_doc to a Task class instance

saved_task = json.loads(saved_doc)

saved_task.id = key #manually map the key to the id

if request.method == "POST":

#update the editable fields

saved_task.description = request.form["description"]

saved_task.is_complete = request.form["is_complete"]

json_doc = json.dumps(task)

client.set(key, json_doc)

return redirect(url_for("index"))

return render_view("edit.html", model=saved_task)

Regardless of which language or client you are using, the basic pattern will be the same for all editing scenarios. You start by getting the id parameter from the request, and use that parameter to look up the saved document via the key/value get operation. Then you convert the JSON document to an instance of a Task class.

If you are showing the form to edit the task (an HTTP GET request), then you'll pass that task to the form so that it can be rendered with prefilled data. The following Flask Jinja2 template demonstrates how this process works:

<div>Description:

</div>

<div>Complete:

</div>

<div>

</div>

</form>

To round out our CRUD task list, we need to include an option to delete tasks. We'll keep this feature simple and leave the "Are you sure you want to delete this item?" alert that typically precedes such an action. We'll start by rearranging the table that displays our task list so that an Edit and a Delete link appear to the right of each task:

{% for item in model %}

<tr>

<td>{{ item.description }}</td>

<td>{{ item.is_complete }}</td>

<td>

<a href="/tasks/delete/{{ item.id }}">Delete</a>

</tr>

{% endfor %}

When a user clicks on Delete, the delete action will be called on the server, which will get the id property from the query string and then remove the item using the key/value API:

@app.route("/tasks/delete/<key>", methods=["GET"])

def delete(key):

client.delete(key)

return redirect(url_for("index"))

Again, you'd typically not delete an item so freely over an HTTP GET request. The important point here is to understand that when you remove an item, it is always deleted by its key. There is no Couchbase equivalent of SQL's DELETE FROM Table WHERE Column = 'VALUE' statement. If you need to remove an item based on another value, you'll have to create a view to find that value's key and then remove it.

Showing only incomplete tasks

If we want to include a list view that displays only tasks that are not yet marked as complete, we'll need to modify our view to incorporate this change. The following snippet shows this modification. The action and view for the corresponding list page differ only in the name of the Couchbase view queried by the client (for example, all_incomplete and all):

//view named "all_incomplete" in the "tasks" design document

function(doc, meta) {

if (doc.type == "task" && doc.isComplete === false) {

emit(null, null);

}

Notice that the check for incomplete status explicitly uses JavaScript's === operator. If you haven't used this operator, you should now know that it performs an explicit type check along with a value check. The reason to use it here is that ! doc.isComplete would return false if the property is undefined (which might be acceptable in this particular case, but not in most other cases).

Alternatively, we can create a view where the isComplete property is indexed, allowing us to list complete, incomplete, or all tasks. To do so, we'll simply emit the isComplete property instead of null for the view's key:

function(doc, meta) {

if (doc.type == "task") {

emit(doc.isComplete, null);

}

When the query is made to the task list, the desired complete status is either included or omitted entirely:

#find all complete

client.query("tasks", "by_status", key=true)

#find all incomplete

client.query("tasks", "by_status", key=false)

#find all

client.query("tasks", "by_status")

Nested tasks

To make our simple task list app a little more interesting, we'll add the ability to nest tasks. In other words, we'll allow some tasks to be subtasks of other tasks. Doing so requires only a slight change to our model, for example adding a ParentId property:

public class Task

{

public string Description { get; set; }

public bool IsComplete { get; set; }

public string ParentId { get; set; }

public bool Type { get { return "task"; } }

}

There are numerous ways to set up a user interface to allow parent tasks to be set. In the interest of brevity, we'll assume that our create and edit actions and views have two simple additions:

#get all tasks returned by the

#all_incomplete view in the tasks design document

tasks = client.query("tasks", "all_incomplete")

#when saving tasks, assign the parentId

task.parent_id = request.form["parentId"]

#pass the tasks to the view

#model passed only for edit

return render_template("index.html", model=task, tasks=tasks)

{% for task in tasks %}

<option value="task.id">task.Description</option>

{% endfor %}

</select>

The preceding snippet demonstrates that we'll need to query for all incomplete tasks, pass those tasks to the view for use as the data in an HTML select element, and assign the selected value back to the task we save on posting the data back.

To view tasks and their children, we'll need to write a view that groups related documents. The approach we'll use will be similar to the example in Chapter 5, Introducing N1QL, where we created a collated view to show breweries and their beer types. The only real difference here is that we have a single type of document with a reference to itself:

function(doc, meta) {

if (doc.type == "task") {

if (! doc.parentId || (doc.parentId == "")) {

emit([meta.Id, 0], null);

} else {

emit([doc.parentId, 1], null);

}

In the preceding map function, we first perform the standard type check. Next, we check whether a document has a parentId property. If it does, we check whether it's an empty string. If there's no valid parent ID, we assume this is a parent document (either zero or more children). The parent's key is emitted to the view index. If a document is a child (that is, it has a parentId property), then its key (meta.id) is never indexed, only parentId will be indexed.

The consequence of this view is that all parent documents will appear first, followed immediately by their child tasks (if any). Recall that this ordering is the result of Couchbase sorting the views based on emitted keys. By emitting 0 for the parent and 1 for all children, we guarantee that the parent will always appear first.

To build a page that displays a task with its children listed, we can simply query the view using an array key, where the first element is the parent's ID (or key) and the second element is the number 1 we used to identify children in our map function, as shown next. The following Python snippet omits some details, including the JSON conversions previously shown:

@app.route("/tasks/view/<key>", methods=["GET"])

def edit(key):

parent = client.get(key) #retrieve from the bucket

children = client.query("tasks", "with_children", key=[key, 1])

return render_view("view.html", parent=parent, children=children)

Summary

In this chapter, we walked through the basics of creating a simple Couchbase-backed application. In more complex applications, we need to concern ourselves with advanced tasks, such as locking records with CAS or general design patterns for a particular platform.

What you did learn, however, were the basic building blocks of a Couchbase application. All database-driven applications start with some simple form of CRUD, and grow more complex when the requirements are fleshed out. With the topics covered in this chapter, you'll be able to start building an application with Couchbase.

An important thing to remember about building a Couchbase application is that by virtue of being key/value stores, Couchbase applications tend to be simpler in terms of data management. Effectively, all changes to data occur one-at-a time and by key.

Moreover, documents are retrieved in full and updated in full. There are no partial updates. While this might seem like a limiting feature, it does reduce a fair deal of friction found when working with other data stores, where object mappings are made more complex by joins, projections, and aggregations.

Because Couchbase supports so many programming platforms with its SDKs, it would have been impossible to visit all of them in this chapter's examples. For those who wish to see more complete examples of the code in this chapter, the source for working applications will be available at https://bitbucket.org/johnzablocki/.