Learning Web Application Development (2014)
Chapter 8. The Platform
At this point, we roughly know how to build a web application using client- and server-side technologies, but our application is not going to see much use if it’s only running on our computer. The next piece of the puzzle is sharing our application by getting it up and running on the Web.
In the past, making this happen required a tremendous amount of work—we’d have to purchase server space with a fixed IP address, install and configure the required software, purchase a domain name, and then point the domain name to our application. Fortunately, times have changed and these days there is a category of hosting services called Platforms-as-a-Service (PaaS) that takes care of the hard work for us.
You may have heard the term cloud computing; this related concept is based around the idea that low-level details of software maintenance and computation can and should be moved off of local computers and onto the Web. With a PaaS, we don’t need to know any of the details of web-server management and configuration, and can focus more on just getting our applications to work correctly. In other words, a PaaS is the type of technology that the cloud computing model enables.
In this chapter, we’ll learn how to deploy our web applications on an open source PaaS called Cloud Foundry. Although we’ll focus on Cloud Foundry, most of the concepts that we’ll learn can be generalized to other PaaS services like Heroku or Nodejitsu.
Cloud Foundry
Cloud Foundry is an open source PaaS originally developed by VMware. You can read more about the service on its home page, located at http://cloudfoundry.com and shown in Figure 8-1. It offers a free 60-day trial that you can use to get some of the sample apps in this book deployed.
Figure 8-1. The Cloud Foundry home page
Creating an Account
To start, you’ll need to create an account on http://www.cloudfoundry.com. You can access the 60-day free trial by clicking the link in the upper-right corner and typing in your email address. You will get a response with information on how to set up your account.
Getting Your App Ready for Deployment
To deploy applications, you’ll need a tool appropriately called cf. Installing that on your local machine requires you to first set up the Ruby language and the RubyGems package manager. I’ve also included the cf app as part of the node-dev-bootstrap project, so you’ll be able to deploy directly from your guest machine without any additional setup!
To get started, set up a Chapter8 directory by recloning the node-dev-bootstrap project as you did in the previous two chapters. Once you do that, start your guest machine and SSH into it.
We’ll start by deploying the basic server.js program that comes with node-dev-bootstrap, but before we can do that we’ll need to do two things. First, we’ll need to add a package.json file because Cloud Foundry expects that file in all apps that we’re going to deploy. Because our default server app doesn’t depend on any external Node.js modules, it’s very short:
{
"name":"sp_example",
"description": "My first Cloud Foundry app!"
}
Next, we’ll need to make a minor modification to the default server.js file that’s included with the node-dev-bootstrap project. This change involves the port number on which our server listens: for technical reasons, Cloud Foundry has to assign us a port number for our code to listen on, but we often won’t know that port until our code is running. Fortunately, Cloud Foundry delivers the port number to us via an environment variable called PORT. We can access it through the process.env.PORT variable in our Node.js program:
var http = require("http"),
// if the environment PORT is set up,
// listen on that one, otherwise listen
// on port 3000
port = process.env.PORT || 3000;
var server = http.createServer(function (req, res) {
res.writeHead(200, {"Content-Type": "text/plain"});
res.end("Hello from Cloud Foundry!");
});
server.listen(port);
console.log("Server listening on port " + port);
Once again, we see the || idiom that was introduced in Chapter 7. Basically, this code says the following: if process.env.PORT is defined, use it; otherwise, use 3000. This lets us set up our port number so that our application works correctly on both our guest machine and when we get it deployed on Cloud Foundry.
Deploying Our App
At this point, we should be able to run it on our virtual machine and access it from our browser as we have in the past. And now that we’ve created a package.json file and set up our app to listen on the correct port, we are also ready to deploy it to Cloud Foundry.
As I mentioned before, that requires us to use the cf program, which is already installed on our guest machine. So to start, enter the directory that contains the server.js file on your guest machine.
The Cloud Foundry platform API is located at api.run.pivotal.io. So we start by telling cf where our target Cloud Foundry platform is located using the target subcommand:
vagrant $ cf target api.run.pivotal.io
Setting target to https://api.run.pivotal.io... OK
Next, we’ll log in with the login subcommand and type in our credentials as we set them up when we created our Cloud Foundry account. Upon logging in, cf will ask which deployment space we want to use. I typically use the development space when I’m experimenting:
vagrant $ cf login
target: https://api.run.pivotal.io
Email> me@semmy.me
Password> ************
Authenticating... OK
1: development
2: production
3: staging
Space> 1
Switching to space development... OK
If everything has gone smoothly, our next step is to push our app to Cloud Foundry! How can we do that? It’s as easy as using the push subcommand. Along with the push subcommand, we’ll need to include the command that launches our app:
vagrant $ cf push --command "node server.js"
The push subcommand enters us into a dialogue with Cloud Foundry. It will ask us a few questions about our application and the environment we’ll use. Here is the dialogue I had with Cloud Foundry—yours should look similar for this example:
Name> sp_example
Instances> 1
1: 128M
2: 256M
3: 512M
4: 1G
Memory Limit> 256M
Creating sp_example... OK
1: sp_example
2: none
Subdomain> sp_example
1: cfapps.io
2: none
Domain> cfapps.io
Creating route sp_example.cfapps.io... OK
Binding sp_example.cfapps.io to sp_example... OK
Create services for application?> n
Save configuration?> n
Uploading sp_example... OK
Preparing to start sp_example... OK
-----> Downloaded app package (4.0K)
-----> Resolving engine versions
WARNING: No version of Node.js specified in package.json, see:
https://devcenter.heroku.com/articles/nodejs-versions
Using Node.js version: 0.10.21
Using npm version: 1.2.30
-----> Fetching Node.js binaries
-----> Vendoring node into slug
-----> Installing dependencies with npm
npm WARN package.json sp_example@ No repository field.
npm WARN package.json sp_example@ No readme data.
Dependencies installed
-----> Building runtime environment
-----> Uploading droplet (15M)
Checking status of app 'sp_example'...
1 of 1 instances running (1 running)
Push successful! App 'sp_example' available at sp_example.cfapps.io
WARNING
The name of your app has to be unique among all names on that domain. That means if you name your app something like example, it’s likely that you’ll get a failure when you try to deploy. I attempt to avoid that issue by adding my initials and an underscore to the beginning of my name. This doesn’t always work, so you may have to find other approaches to generating unique app names.
If everything went smoothly, your app is actually running on the Web now! We can confirm this by opening up a web browser and navigating to the URL that cf gave us (in my case, it’s http://sp_example.cfapps.io). Once you do that, you should see a response from your app.
Getting Information About Your Apps
Now that your app is up and running, you can use other subcommands of cf to get information about the status of your applications. For example, you use the apps sub-command to get a list of all of your applications on Cloud Foundry and their statuses:
vagrant $ cf apps
name status usage url
sp_example running 1 x 256M sp_example.cfapps.io
One major issue with running your application on a PaaS is that you can’t see the results of your console.log statements as easily as you can when running your application locally. This can be a major problem if your program crashes and you need to determine why. Fortunately, Cloud Foundry offers the logs subcommand that you can use on your running apps to view your program’s logs:
vagrant $ cf logs sp_example
Getting logs for sp_example #0... OK
Reading logs/env.log... OK
TMPDIR=/home/vcap/tmp
VCAP_APP_PORT=61749
USER=vcap
VCAP_APPLICATION= { ... }
PATH=/home/vcap/app/bin:/home/vcap/app/node_modules/.bin:/bin:/usr/bin
PWD=/home/vcap
VCAP_SERVICES={}
SHLVL=1
HOME=/home/vcap/app
PORT=61749
VCAP_APP_HOST=0.0.0.0
MEMORY_LIMIT=256m
_=/usr/bin/env
Reading logs/staging_task.log... OK
-----> Downloaded app package (4.0K)
-----> Resolving engine versions
WARNING: No version of Node.js specified in package.json, see:
https://devcenter.heroku.com/articles/nodejs-versions
Using Node.js version: 0.10.21
Using npm version: 1.2.30
-----> Fetching Node.js binaries
-----> Vendoring node into slug
-----> Installing dependencies with npm
npm WARN package.json sp_example@ No repository field.
npm WARN package.json sp_example@ No readme data.
Dependencies installed
-----> Building runtime environment
Reading logs/stderr.log... OK
Reading logs/stdout.log... OK
Server listening on port 61749
You’ll see that Cloud Foundry prints the contents of four logs that are stored. The first is env.log, which has all of your environment variables that you can access via the process.env variable in your program. The second is staging_task.log, which logs everything that happens when your program first starts (you’ll note that this is the same content that is printed out when you first run cf push. The last two are stderr.log and stdout.log. You’ll see that stdout.log includes the console.log statement that you used in your program. If you use console.err instead, your message will appear in stderr.log.
Updating Your App
You can easily send a newer version of your app to Cloud Foundry by pushing it again. Modify your server.json so that it returns a little more information:
var http = require("http"),
port = process.env.PORT || 3000;
var server = http.createServer(function (req, res) {
res.writeHead(200, {"Content-Type": "text/plain"});
res.write("The server is running on port " + port);
res.end("Hello from Cloud Foundry!");
});
server.listen(port);
console.log("Server listening on port " + port);
Once you make that change, you can run the push subcommand again along with the name of the app you’d like to update. This will push the new changes without requiring you to answer all of the questions again:
vagrant $ cf push sp_example
Save configuration?> n
Uploading sp_example... OK
Stopping sp_example... OK
Preparing to start sp_example... OK
-----> Downloaded app package (4.0K)
-----> Downloaded app buildpack cache (4.0K)
-----> Resolving engine versions
WARNING: No version of Node.js specified in package.json, see:
https://devcenter.heroku.com/articles/nodejs-versions
Using Node.js version: 0.10.21
Using npm version: 1.2.30
-----> Fetching Node.js binaries
-----> Vendoring node into slug
-----> Installing dependencies with npm
npm WARN package.json sp_example@ No repository field.
npm WARN package.json sp_example@ No readme data.
Dependencies installed
-----> Building runtime environment
-----> Uploading droplet (15M)
Checking status of app 'sp_example'...
1 of 1 instances running (1 running)
Push successful! App 'sp_example' available at sp_example.cfapps.io
Deleting Apps from Cloud Foundry
Occasionally, we’ll want to delete apps from Cloud Foundry, particularly when we’re just experimenting. To do that, we can use the delete subcommand:
vagrant $ cf delete sp_example
Really delete sp_example?> y
Deleting sp_example... OK
Dependencies and package.json
In previous examples, our apps have had external dependencies like the express, redis, mongoose, and ntwitter modules. Using basic modules that don’t connect to external services (like express and ntwitter) is pretty straightforward. Because we typically don’t push ournode_modules directory, we simply need to make sure that all of our dependencies are listed in our package.json file.
For example, consider one of our first Express apps. After some minor modifications that get it listening on the correct Cloud Foundry port, it looks like this:
var express = require("express"),
http = require("http"),
app = express(),
port = process.env.PORT || 3000;;
http.createServer(app).listen(port);
console.log("Express is listening on port " + port);
app.get("/hello", function (req, res) {
res.send("Hello World!");
});
app.get("/goodbye", function (req, res) {
res.send("Goodbye World!");
});
We’ll need to include our Express module dependency in our package.json file, which we can do exactly as we did in our previous package.json example:
{
"name": "sp_express",
"description": "a sample Express app",
"dependencies": {
"express": "3.4.x"
}
}
You’ll see that I specified that our app depends on the Express module, specifically any version number that starts with 3.4 (the x is a wildcard). This tells Cloud Foundry which version to install to make our app run correctly. Once we’ve included dependencies in our package.json file, we can push this to Cloud Foundry using the same command we used before:
vagrant $ ~/app$ cf push --command "node server.js"
Name> sp_expressexample
Once we do that, we can go to http://sp_expressexample.cfapps.io/hello or http://sp_expressexample.cfapps.io/goodbye to see our app respond!
Getting our Twitter app or our Amazeriffic app working is a little more challenging because they are dependent on other data storage services—specifically Redis and MongoDB. That means we’ll need to create the services and then set up our app to use them.
Binding Redis to Your App
When we run our app on our virtual machine, services like Redis and MongoDB are running locally. That changes a little when we run our app on a PaaS. Sometimes the services run on the same host, but other times they are run on another hosting service.
Either way, you’ll need to start by setting up the service that you want to run in your interactions with cf. In this section, we’ll set up Redis on Cloud Foundry and then get our tweet counter app connecting to it.
We’ll start by copying our Twitter app from Chapter 7 to our Chapter8/app directory. Make sure your package.json file exists and includes the redis and ntwitter dependencies. Mine looks like this:
{
"name": "tweet_counter",
"description": "tweet counter example for learning web app development",
"dependencies": {
"ntwitter":"0.5.x",
"redis":"0.9.x"
}
}
We’ll also want to update server.js so that it listens on the port specified in process.env.PORT. Once we do that, we can try to push our app! I’ll go ahead and mention ahead of time that this will fail, but it’ll give us the opportunity to set up our Redis service:
vagrant $ cf push --command "node server.js"
Name> sp_tweetcounter
Instances> 1
1: 128M
2: 256M
3: 512M
4: 1G
Memory Limit> 256M
Creating sp_tweetcounter... OK
1: sp_tweetcounter
2: none
Subdomain> sp_tweetcounter
1: cfapps.io
2: none
Domain> cfapps.io
Binding sp_tweetcounter.cfapps.io to sp_tweetcounter... OK
Create services for application?> y
1: blazemeter n/a, via blazemeter
2: cleardb n/a, via cleardb
3: cloudamqp n/a, via cloudamqp
4: elephantsql n/a, via elephantsql
5: loadimpact n/a, via loadimpact
6: mongolab n/a, via mongolab
7: newrelic n/a, via newrelic
8: rediscloud n/a, via garantiadata
9: sendgrid n/a, via sendgrid
10: treasuredata n/a, via treasuredata
11: user-provided , via
Notice that here we have told Cloud Foundry that we’d like to create a service for our application:
What kind?> 8
Name?> rediscloud-dfc38
1: 20mb: Lifetime Free Tier
Which plan?> 1
Creating service rediscloud-dfc38... OK
Binding rediscloud-dfc38 to sp_tweetcounter... OK
Create another service?> n
Bind other services to application?> n
Save configuration?> n
Finish out the dialog and you’ll find that your push will fail. That’s because we haven’t told our app how to connect to the external Redis service yet. The logs should clue us in to that. In fact, when running cf logs sp_tweetcounter, I should see something like the following in mylogs/stderr.log:
Reading logs/stderr.log... OK
events.js:72
throw er; // Unhandled 'error' event
^
Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED
at RedisClient.on_error (/home/vcap/app/node_modules/redis/index.js:189:24)
at Socket.<anonymous> (/home/vcap/app/node_modules/redis/index.js:95:14)
at Socket.EventEmitter.emit (events.js:95:17)
at net.js:441:14
at process._tickCallback (node.js:415:13)
In our app, we don’t send any arguments to Redis, so it attempts to connect on the local server (127.0.0.1), along with the default port number (6379). We need to tell it to connect to the Redis cloud service on the host and port that Cloud Foundry provides for us. Where does Cloud Foundry give us that information? Just like the port number, it gives us information about our bound services in the process.env variable!
If you’ve already typed cf logs sp_tweetcounter, you can scroll up to see the environment variables and look at the contents of the VCAP_SERVICES variable. In the logs, it’s one long JSON string, so I’ve formatted it here to look a little more readable:
VCAP_SERVICES = {
"rediscloud-n/a": [{
"name": "rediscloud-dfc38",
"label": "rediscloud-n/a",
"tags": ["redis", "key-value"],
"plan": "20mb",
"credentials": {
"port": "18496",
"hostname": "pub-redis-18496.us-east-1-4.2.ec2.garantiadata.com",
"password": "REDACTED"
}
}]
}
It includes all of the information that we need to connect to a remote Redis service, including a URL, a port, and a password. If we were so inclined, we could easily code these Redis credentials directly into our program. But it’s probably better to do it programmatically in the same way that we set up the port number.
WARNING
One minor gotcha is that the VCAP_SERVICES environment variable is stored as a string, so we have to use the JSON.parse command in the same way that we did in Chapter 5 to convert it into an object that we can access like any other JavaScript object.
In short, we can get Redis working by modifying the first part of our tweet_counter.js module to look like this:
var ntwitter = require("ntwitter"),
redis = require("redis"), // require the redis module
credentials = require("./credentials.json"),
redisClient,
counts = {},
twitter,
services,
redisCredentials;
// create our twitter client
twitter = ntwitter(credentials);
// set up our services if the environment variable exists
if (process.env.VCAP_SERVICES) {
// parse the JSON string
services = JSON.parse(process.env.VCAP_SERVICES);
redisCredentials = services["rediscloud-n/a"][0].credentials;
} else {
// otherwise we'll set up default values
redisCredentials = {
"hostname": "127.0.0.1",
"port": "6379",
"password": null
};
}
// create a client to connect to Redis
client = redis.createClient(redisCredentials.port, redisCredentials.hostname);
// authenticate
client.auth(redisCredentials.password);
In this code snippet, we’re checking to see if the VCAP_SERVICES environment variable exists, and if it does we’re parsing the string associated with it to produce a JavaScript object from the JSON representation. Next, we’re pulling out the credentials associated with the rediscloud-n/aproperty of the services object. That object is itself an array (because our app could be associated with multiple Redis instances) so we’re getting the first element out of the array.
If the VCAP_SERVICES environment variable is not defined, we’re setting up a redisCredentials object that has the default values. After that, we’re connecting to Redis by specifying the port and hostname, and then we’re sending along the password. If we’re connected to the local Redis instance on our virtual machine, we send null for the password because it doesn’t require us to authenticate.
If we’ve already pushed the app once and it failed, we can simply push it again to update it. This time, I’ll push it using the name that I gave it previously. If I’ve forgotten it, I can always use cf apps to see a list of my apps:
vagrant $ cf apps
Getting applications in development... OK
name status usage url
sp_example running 1 x 256M sp_example.cfapps.io
sp_express running 1 x 256M sp_express.cfapps.io
sp_tweetcounter running 1 x 256M sp_tweetcounter.cfapps.io
vagrant $ cf push sp_tweetcounter
If everything is set up properly, you’ll see the app successfully pushed and you should be able to access it at the URL that Cloud Foundry gives you!
Binding MongoDB to Your App
Getting MongoDB up and running in Cloud Foundry is almost as easy as getting Redis running. Let’s start by copying our Amazeriffic app from Chapter7 into our current directory. We’ll have to make a few modifications that follow the same pattern as before.
First, we’ll change it so it listens on process.env.PORT if that value exists. The code for that is identical to the examples in the preceding section.
After that, we’ll need to get our MongoDB credentials out of process.env.VCAP_SERVICES. That code will look very similar to the Redis code. The main difference is that our MongoDB credentials are all contained in a single string—the uri:
// don't forget to declare mongoUrl somewhere above
// set up our services
if (process.env.VCAP_SERVICES) {
services = JSON.parse(process.env.VCAP_SERVICES);
mongoUrl = services["mongolab-n/a"][0].credentials.uri;
} else {
// we use this when we're not running on Cloud Foundry
mongoUrl = "mongodb://localhost/amazeriffic"
}
Once we get that working, we can push our app to Cloud Foundry as we have before. One minor change is that we’ll need to set up the MongoLab service:
Create services for application?> y
1: blazemeter n/a, via blazemeter
2: cleardb n/a, via cleardb
3: cloudamqp n/a, via cloudamqp
4: elephantsql n/a, via elephantsql
5: loadimpact n/a, via loadimpact
6: mongolab n/a, via mongolab
7: newrelic n/a, via newrelic
8: rediscloud n/a, via garantiadata
9: sendgrid n/a, via sendgrid
10: treasuredata n/a, via treasuredata
11: user-provided , via
What kind?> 6
Name?> mongolab-8a0f4
1: sandbox: 0.5 GB
Which plan?> 1
Creating service mongolab-8a0f4... OK
Once the Mongo service is created, we simply bind it to our application like we did with Redis. As long as our code is ready to connect to the remote service via the URL that is provided to us by the VCAP_SERVICES environment variable, our program should launch exactly as we’d expect.
Summary
In this chapter, we learned how to use Cloud Foundry to get our applications on the Web. Cloud Foundry is an example of a Platform as a Service. A PaaS is a cloud computing technology that abstracts server setup, administration, and hosting, usually through a command-line program or a web interface.
To get our applications running on Cloud Foundry (or any other PaaS), we typically have to make some minor modifications to the code so that it behaves differently depending on whether we’re running it locally in our development environment or in production on Cloud Foundry’s servers.
Cloud Foundry provides external services like Redis and MongoDB as well. To get our applications running with these services, it is necessary for us to both create the service with the cf program, and then bind our application to it.
More Practice and Further Reading
Poker API
In the practice sections of the previous chapters, we created a simple poker API that would determine poker hands. In the last chapter, we added a database component to that application. If you have that working, try modifying it and getting it deployed on Cloud Foundry.
Other Platforms
There is no shortage of cloud platforms these days. One of the most popular ones is Heroku. It allows you to sign up for free without a credit card. If you want to add Redis or Mongo, however, you’ll be required to provide one even though they have free tiers for both of these services.
If you want more practice, try reading Heroku’s Node.js deployment documentation and getting one or more of our apps deployed on its service. I would also encourage you to try others, including Nodejitsu and Microsoft’s Windows Azure. They are all a little different, so comparing and contrasting the features of each is a great learning exercise.