MEAN Machine - A beginner's practical guide to the JavaScript stack (2015)
Build a RESTful Node API
What is REST?
For a little bit on RESTful APIs, take a look at this great presentation: Teach a Dog to REST.
RESTful APIs are becoming a standard across services on the web. It’s not enough to just build an application now, we’re moving towards platforms that can integrate with multiple devices and other websites.
Interconnectivity is quickly becoming the name of the game. With projects like IFTTT and Zapier gaining popularity, users have shown that they like their applications connected. This means that all of their applications have a standard way of “talking” to one another. IFTTT lets you set triggers and responses like “When I write a Tweet, share it as my Facebook status”. The APIs of Twitter and Facebook allow us to do this.
IFTTT
The big players in the game like Facebook and Google have their own APIs that are used across multiple third party apps. You can even find their API Explorers where you can take their API for a test drive and see what information you can pull from the API. For instance, once you sign in, you can grab the data for all your friends from the API Explorer. Using APIs has quickly become the standard for building applications.
It’s not enough to have an application be standalone, you must build a website and multiple mobile apps using the same data. If your application takes off, then you’ll want users to be able to build third-party applications on your data. This is where your API comes in. We’ll learn how to build one using Node so that multiple applications can use that API and your data.
Backend Services for our Angular Frontend
In this chapter, we’ll be looking at creating a RESTful API using Node, Express 4 and its Router, and Mongoose to interact with a MongoDB instance. We will also be testing our API using Postman in Chrome or RESTClient in Firefox.
This will be a completely new sample application from our previous chapters and we will be using it in a few chapters when we get to our Angular frontend. Let’s take a look at the API we want to build and what it can do.
Sample Application
Let’s say we are going to build a CRM (Customer Relations Management) tool. This would mean that we would need to be able to manage and handle CRUD on the users in our database. Users will be the main thing we will focus on when building this API and in a few chapters, Angular will build the frontend views that will access our Users API.
Let’s build an API that will:
· Handle CRUD for an item (users)
· Have a standard URL (http://example.com/api/users and http://example.com/api/users/:user_id)
· Use the proper HTTP verbs to make it RESTful (GET, POST, PUT, and DELETE)
· Return JSON data
· Log all requests to the console
Getting Started
Let’s look at all the files we will need to create our API. We will need to define our Node packages, start our server using Express, define our model, declare our routes using Express, and last but not least, test our API.
Here is our file structure. We won’t need many files and we’ll keep this very simple for demonstration purposes. When moving to a production or larger application, you’ll want to separate things out into a better structure (like having your routes in their own file). We’ll go over file structure and best practices later in the book.
1 - app/
2 ----- models/
3 ---------- user.js // our user model
4 - node_modules/ // created by npm. holds our dependencies/packages
5 - package.json // define all our node app and dependencies
6 - server.js // configure our application and create routes
Defining our Node Packages (package.json)
As with all of our Node projects, we will define the packages we need in package.json. Go ahead and create that file with these packages.
1 {
2 "name": "node-api",
3 "main": "server.js",
4 "dependencies": {
5 "morgan": "~1.5.0",
6 "express": "~4.10.3",
7 "body-parser": "~1.9.3",
8 "mongoose": "~3.8.19",
9 "bcrypt-nodejs": "0.0.3"
10 }
11 }
As a shortcut, you could type the following into your command line and they would add themselves to the package.json file as well:
npm install express morgan mongoose body-parser bcrypt-nodejs --save
What do these packages do?
· express is the Node framework.
· morgan allows us to log all requests to the console so we can see exactly what is going on.
· mongoose is the ODM we will use to communicate with our MongoDB database.
· body-parser will let us pull POST content from our HTTP request so that we can do things like create a user.
· bcrypt-nodejs will allow us to hash our passwords since it is never safe to store passwords plaintext in our databases
Install the Node Packages
This might be the easiest step. Go into the command line in the root of your application and type:
1 npm install
npm will now pull in all the packages defined into a node_modules folder in our project.
Node’s package manager will bring in all the packages we defined in package.json. Simple and easy. Now that we have our packages, let’s go ahead and use them when we set up our API.
We’ll be looking to our server.js file to setup our app since that’s the main file we declared in package.json.
Setting Up Our Server (server.js)
Node will look here when starting the application so that it will know how we want to configure our application and API.
We will start with the bare essentials necessary to start up our application. We’ll keep this code clean and commented well so we understand exactly what’s going on every step of the way.
1 // BASE SETUP
2 // ======================================
3
4 // CALL THE PACKAGES --------------------
5 var express = require('express'); // call express
6 var app = express(); // define our app using express
7 var bodyParser = require('body-parser'); // get body-parser
8 var morgan = require('morgan'); // used to see requests
9 var mongoose = require('mongoose'); // for working w/ our database
10 var port = process.env.PORT || 8080; // set the port for our app
11
12 // APP CONFIGURATION ---------------------
13 // use body parser so we can grab information from POST requests
14 app.use(bodyParser.urlencoded({ extended: true }));
15 app.use(bodyParser.json());
16
17 // configure our app to handle CORS requests
18 app.use(function(req, res, next) {
19 res.setHeader('Access-Control-Allow-Origin', '*');
20 res.setHeader('Access-Control-Allow-Methods', 'GET, POST');
21 res.setHeader('Access-Control-Allow-Headers', 'X-Requested-With,content-type, \
22 Authorization');
23 next();
24 });
25
26 // log all requests to the console
27 app.use(morgan('dev'));
28
29 // ROUTES FOR OUR API
30 // =============================
31
32 // basic route for the home page
33 app.get('/', function(req, res) {
34 res.send('Welcome to the home page!');
35 });
36
37 // get an instance of the express router
38 var apiRouter = express.Router();
39
40 // test route to make sure everything is working
41 // accessed at GET http://localhost:8080/api
42 apiRouter.get('/', function(req, res) {
43 res.json({ message: 'hooray! welcome to our api!' });
44 });
45
46 // more routes for our API will happen here
47
48 // REGISTER OUR ROUTES -------------------------------
49 // all of our routes will be prefixed with /api
50 app.use('/api', apiRouter);
51
52 // START THE SERVER
53 // ===============================
54 app.listen(port);
55 console.log('Magic happens on port ' + port);
Wow we did a lot there! It’s all very simple though so let’s walk through it a bit.
Base Setup: In our base setup, we pull in all the packages we pulled in using npm. We’ll grab express, mongoose, define our app, get bodyParser and configure our app to use it. We can also set the port for our application and grab the user model that we will define later.
We are setting our configuration to allow requests from other domains to prevent CORS errors. This allows any domain to access our API.
Routes for Our API: This section will hold all of our routes. The structure for using the Express Router let’s us pull in an instance of the router. We can then define routes and then apply those routes to a root URL (in this case, API). There is also a home page route to say hello; this is just a basic route to make sure that everything is working.
Start the Server: We’ll have our express app listen to the port we defined earlier. Then our application will be live and we can test it!
Starting the Server and Testing
Let’s make sure that everything is working up to this point. We will start our Node app and then send a request to the one route we defined to make sure we get a response.
Let’s start our server. From the command line, type:
1 nodemon server.js
You should see your Node app start up and Express will create a server.
Starting Node Server
Now that we know our application is up and running, let’s test it. You should be able to see our message returned by visiting the basic route: http://localhost:8080. Now let’s test the rest of our routes (the API routes).
Basic Route
Testing Our API Using Postman
Now navigate to http://localhost:8080/api in your browser and you will see our JSON message. Seeing it is great and all, but the best way to test out our API is to use a tool built for this sort of thing. Postman will help us test our API. It will basically send HTTP requests to a URL of our choosing. We can even pass in parameters (which we will soon).
Open up Postman in Chrome and let’s walk through how to use it.
Postman REST Client Node API
All you have to do is enter your request URL, select an HTTP verb, and click Send. Simple enough right? We can even see a history of the past calls we’ve made so that we can switch back and forth between calls we need to test. Think creating and deleting records on the fly.
Here’s the moment we’ve been waiting for. Does our application work the way we configured it? Enter http://localhost:8080/api into the URL. GET is what we want since we just want to get data. Now click Send.
Node API Postman Test
Sweet! We got back exactly what we wanted. Now we know we can serve information to requests. Let’s wire up our database so we can start performing CRUD operations on some users.
Database and User Model
We’ll keep this short and sweet so that we can get to the fun part of building the API routes. All we need to do is create a MongoDB database and have our application connect to it. We will also need to create a user mongoose model so we can use mongoose to interact with our database.
Creating Our Database and Connecting
We will be using a database provided by Modulus. You can definitely create your own database and use it locally or use online database providers like Modulus or Mongolab.
Once you have your database created and have the URI to connect to (mongodb://node:node@novus.modulusmongo.net:27017/Iganiq8o), let’s add it to our application. In server.js in the Base Setup section, let’s add these two lines.
1 // server.js
2
3 // BASE SETUP
4 // ============================
5
6 ...
7
8 // connect to our database (hosted on modulus.io)
9 mongoose.connect('mongodb://node:noder@novus.modulusmongo.net:27017/Iganiq8o');
10
11 ...
All you need is a URI like above so that your application can connect. This URI contains the host (novus.modulusmongo.net), the port (27017), the database (Iganiq8o), and the user and password (node@noder).
You can also create your MongoDB instance locally like we did in Chapter 8. Your mongoose.connect() url would then be: mongoose.connect(mongodb://localhost:27017/myDatabase.
Since we already grabbed the mongoose package earlier with ‘npm install’, we just need to connect to our remote database hosted by Modulus or locally. Now that we are connected to our database, let’s create a mongoose model to handle our users.
User Model (app/models/user.js)
Since the model won’t be the focus of this tutorial, we’ll just create a model and provide our users with a name field. That’s it. Let’s create that file and define the model.
1 // grab the packages that we need for the user model
2 var mongoose = require('mongoose');
3 var Schema = mongoose.Schema;
4 var bcrypt = require('bcrypt-nodejs');
5
6 // user schema
7 var UserSchema = new Schema({
8 name: String,
9 username: { type: String, required: true, index: { unique: true }},
10 password: { type: String, required: true, select: false }
11 });
12
13 // hash the password before the user is saved
14 UserSchema.pre('save', function(next) {
15 var user = this;
16
17 // hash the password only if the password has been changed or user is new
18 if (!user.isModified('password')) return next();
19
20 // generate the hash
21 bcrypt.hash(user.password, null, null, function(err, hash) {
22 if (err) return next(err);
23
24 // change the password to the hashed version
25 user.password = hash;
26 next();
27 });
28 });
29
30 // method to compare a given password with the database hash
31 UserSchema.methods.comparePassword = function(password) {
32 var user = this;
33
34 return bcrypt.compareSync(password, user.password);
35 };
36
37 // return the model
38 module.exports = mongoose.model('User', UserSchema);
There is a lot happening here, but it’s fairly simple. We must create our Schema. We are defining name, username, and password as Strings. By setting the index and unique attributes, we are telling Mongoose to create a unique index for this path. This means that a username can not be duplicated. Another mongoose feature that we will implement is select: false on the password attribute. When we query the list of users or a single user, there will be no need to provide the password. By setting select to false, the password will not be returned when listing our users, unless it is explicitly called.
We are also creating a function using pre that will ensure that the password is hashed before we save the user.
It is always important to make sure that we are not saving plaintext passwords to the database. We have to make sure our applications are as secure as can be from the start.
With mongoose, we are also creating a method on our user model to compare the password with a given hash. This is how we will authenticate our users in Chapter 10.
With that file created, let’s pull it into our server.js so that we can use it within our application. We’ll add one more line to that file and use require like we have before to pull in features.
1 // server.js
2
3 // BASE SETUP
4 // ==========================
5
6 var User = require('./app/models/user');
Now our entire application is ready and wired up so we can start building out our routes. These routes will define our API, which is the main reason this chapter exists. Onward!
Express Router and Routes
We will use an instance of the Express Router to handle all of our API routes. Here is an overview of the routes we will require, what they will do, and the HTTP Verb used to access it.
|
/api/users |
GET |
Get all the users |
|
/api/users |
POST |
Create a user |
|
/api/users/:user_id |
GET |
Get a single user |
|
/api/users/:user_id |
PUT |
Update a user with new info |
|
/api/users/:user_id |
DELETE |
Delete a user |
This will cover the basic routes needed for an API. This also keeps to a good format where we have kept the actions we need to execute (GET, POST, PUT, and DELETE) as HTTP verbs.
Route Middleware
We’ve already defined our first route and seen it in action. The Express Router gives us a great deal of flexibility in defining our routes.
We’ve talked on route middleware in the Node Routing chapter. For this example we are just going to console.log() the request to the console. Let’s add that middleware to our ‘server.js’ file now.
1 // ROUTES FOR OUR API
2 // =============================
3 var apiRouter = express.Router(); // get an instance of the express Router
4
5 // middleware to use for all requests
6 apiRouter.use(function(req, res, next) {
7 // do logging
8 console.log('Somebody just came to our app!');
9
10 // we'll add more to the middleware in Chapter 10
11 // this is where we will authenticate users
12
13 next(); // make sure we go to the next routes and don't stop here
14 });
15
16 // test route to make sure everything is working
17 // (accessed at GET http://localhost:8080/api)
18 apiRouter.get('/', function(req, res) {
19 res.json({ message: 'hooray! welcome to our api!' });
20 });
21
22 // more routes for our API will happen here
23
24 // REGISTER OUR ROUTES -------------------------------
25 // all of our routes will be prefixed with /api
26 app.use('/api', apiRouter);
All we needed to do to declare that middleware was to use router.use(function()). The order of how we define the parts of our router is very important. They will run in the order that they are listed like we talked about earlier.
We are sending back information as JSON data. This is standard for an API and the people using your API will be eternally grateful.
We will also add next() to indicate to our application that it should continue to the other routes or next middleware. This is important because our application would stop at this middleware without it.
Now when we send a request to our application using Postman, the request will be logged to our Node console (the command line).
Middleware Uses Using middleware like this can be very powerful. We can do validations to make sure that everything coming from a request is safe and sound. We can throw errors here in case something in the request is wrong. We can do some extra logging for analytics or any statistics we’d like to keep. And the big usage is to authenticate the API calls by checking if a user has the correct token, which we will do in the next chapter.
Creating the Basic Routes
We will now create the routes to handle getting all the users and creating a user. This will both be handled using the /api/users route. We’ll look at creating a user first so that we have users to work with.
Creating a User POST (/api/users)
We will add the new route to handle POST and then test it using Postman.
1 // ROUTES FOR OUR API
2 // ===============================
3
4 // route middleware and first route are here
5
6 // on routes that end in /users
7 // ----------------------------------------------------
8 apiRouter.route('/users')
9
10 // create a user (accessed at POST http://localhost:8080/api/users)
11 .post(function(req, res) {
12
13 // create a new instance of the User model
14 var user = new User();
15
16 // set the users information (comes from the request)
17 user.name = req.body.name;
18 user.username = req.body.username;
19 user.password = req.body.password;
20
21 // save the user and check for errors
22 user.save(function(err) {
23 if (err) {
24 // duplicate entry
25 if (err.code == 11000)
26 return res.json({ success: false, message: 'A user with that\
27 username already exists. '});
28 else
29 return res.send(err);
30 }
31
32 res.json({ message: 'User created!' });
33 });
34
35 })
Now we have created the POST route for our application. We will use Express’s apiRouter.route() to handle multiple routes for the same URI. We are able to handle all the requests that end in /users.
Let’s look at Postman now to create our user. We are passing the name variable. This is how the information would look if it came from a form on our website or application.
Node API Postman POST Create User
Notice that we are sending the name data as x-www-form-urlencoded. This will send all of our data to the Node server as query strings.
We get back a successful message that our user has been created. Let’s handle the API route to get all the users so that we can see the user that just came into existence.
If the username was already used, mongoose will spit out an error code of 11000. We are going to check for that error code if there is an error and return the message that 'A user with that username already exists..
Duplicate User
Getting All Users (GET /api/users)
This will be a simple route that we will add to ‘server.js’ onto the router.route('/users') we created for the POST. With router.route(), we are able to chain together the different routes using different HTTP actions. This keeps our application clean and organized.
1 // <-- route middleware and first route are here
2
3 // on routes that end in /users
4 // ----------------------------------------------------
5 apiRouter.route('/users')
6
7 // create a user (accessed at POST http://localhost:8080/api/users)
8 .post(function(req, res) {
9 ...
10 })
11
12 // get all the users (accessed at GET http://localhost:8080/api/users)
13 .get(function(req, res) {
14 User.find(function(err, users) {
15 if (err) res.send(err);
16
17 // return the users
18 res.json(users);
19 });
20 });
Straightforward route. Just send a GET request to http://localhost:8080/api/users and we’ll get all the users back in JSON format.
Node API Postman GET All
We can see that Holly is in our database and her password has even been hashed thanks to bcrypt and that function that we created that works whenever we save a user to the database. Remember that _id that was automatically generated for Holly here. We will use it when we create the route to get a single user.
Creating Routes for A Single Item
We’ve handled the group for routes ending in /users. Let’s now handle the routes for when we pass in a parameter like a user’s id.
The things we’ll want to do for this route, which will end in /users/:user_id will be:
· Get a single user.
· Update a user’s info.
· Delete a user.
· The :user_id from the request will be accessed thanks to that body-parser package we called earlier.
Getting a Single User (GET /api/users/:user_id)
We’ll add another apiRouter.route() to handle all requests that have a :user_id attached to them. Again, drop it into your ‘server.js’ file after the ‘apiRouter.route(‘/users’)’ chunk.
1 // on routes that end in /users/:user_id
2 // ----------------------------------------------------
3 apiRouter.route('/users/:user_id')
4
5 // get the user with that id
6 // (accessed at GET http://localhost:8080/api/users/:user_id)
7 .get(function(req, res) {
8 User.findById(req.params.user_id, function(err, user) {
9 if (err) res.send(err);
10
11 // return that user
12 res.json(user);
13 });
14 })
From our call to get all the users, we can a list of all of our users, each with a unique id. Let’s grab one of those ids and test it with Postman.
Enter ‘http://localhost:8080/api/users/5481e26e93b9bba7171bfd5e’, select GET and hit send.
And this will return our pretty user data. Easy peezy!
1 {
2 "password": "$2a$10$rgraupzsDZHbY77CQSInHuZ90RWvq/NWsi0kOueePetqUBvj2.jKa",
3 "username": "chris",
4 "name": "Chris",
5 "_id": "5481e26e93b9bba7171bfd5e",
6 "__v": 0
7 }
Node API Postman Get Single User
Now that we can grab a user from our API, let’s look at updating a user’s name. Let’s say he wants to be more sophisticated, so we’ll rename him from Klaus to Sir Klaus.
Updating a User’s Info (PUT /api/users/:user_id)
Let’s chain a route onto our this router.route() and add a .put().
1 // on routes that end in /users/:user_id
2 // -------------------------------------
3 apiRouter.route('/users/:user_id')
4
5 // get the user with that id
6 // (accessed at GET http://localhost:8080/api/users/:user_id)
7 .get(function(req, res) {
8 ...
9 })
10
11 // update the user with this id
12 // (accessed at PUT http://localhost:8080/api/users/:user_id)
13 .put(function(req, res) {
14
15 // use our user model to find the user we want
16 User.findById(req.params.user_id, function(err, user) {
17
18 if (err) res.send(err);
19
20 // update the users info only if its new
21 if (req.body.name) user.name = req.body.name;
22 if (req.body.username) user.username = req.body.username;
23 if (req.body.password) user.password = req.body.password;
24
25 // save the user
26 user.save(function(err) {
27 if (err) res.send(err);
28
29 // return a message
30 res.json({ message: 'User updated!' });
31 });
32
33 });
34 })
We will use the given id from the PUT request, grab that user, make changes, and save him back to the database. It’s important to note that we will only change the name, username, or password in the database if it has actually changed.
Node API Postman Update Record
Since we added the conditionals in our PUT route, if the username or password hasn’t been passed through, those fields won’t be updated. We wouldn’t want those updated to be blanks if they weren’t passed in.
We can also use the GET /api/users call we used earlier to see that her name has changed.
Node API Postman All Updated
Holly’s name in the database has now been changed from Holly to her full name, Holly Lloyd. That was all fun while it lasted. Now let’s delete Holly. :(
Deleting a User (DELETE /api/users/:user_id)
When someone requests that a user is deleted, all they have to do is send a DELETE to /api/users/:user_id
Let’s add the code for deleting users.
1 // on routes that end in /users/:user_id
2 // ----------------------------------------------------
3 apiRouter.route('/users/:user_id')
4
5 // get the user with that id
6 // (accessed at GET http://localhost:8080/api/users/:user_id)
7 .get(function(req, res) {
8 ...
9 })
10
11 // update the user with this id
12 // (accessed at PUT http://localhost:8080/api/users/:user_id)
13 .put(function(req, res) {
14 ...
15 })
16
17 // delete the user with this id
18 // (accessed at DELETE http://localhost:8080/api/users/:user_id)
19 .delete(function(req, res) {
20 User.remove({
21 _id: req.params.user_id
22 }, function(err, user) {
23 if (err) return res.send(err);
24
25 res.json({ message: 'Successfully deleted' });
26 });
27 });
Now when we send a request to our API using DELETE with the proper user_id, we’ll delete our user from existence.
Node API Postman Delete
When we try to get all the users you should notice that one is now missing.
Node API Get All Nothing
Conclusion
We now have the means to handle CRUD on a specific resource (our beloved users) through our own API. Using the techniques above will be a good foundation to move into building larger and more robust APIs.
This has been a quick look at creating a Node API using Express 4. There are many more things you can do with your own APIs. You can add authentication, create better error messages, and add different sections so you’re not just working with users.
We’ll be using this API as the data backend of our Angular application that we will start to build. Let’s look at authenticating this API with token based authentication next and then we’ll be moving onto learning AngularJS.