Scaling Applications - Learning OpenShift (2014)

Learning OpenShift (2014)

Chapter 12. Scaling Applications

One of the great things about using Platform as a Service (PaaS), such as OpenShift Online, is the ability to leverage application scaling to satisfy the demands of your users for fast response times. Having an application that can handle all incoming traffic, even under the heaviest of loads, is critical for production quality applications. We all dream of a time when an application we have written is under such heavy usage that adding additional servers to handle the capacity is a requirement. Unless you are extremely familiar with system administration, setting up a clustered environment is not an easy task. In this chapter, we will learn how to leverage the provided scaling functionality inside of OpenShift so that our minds can be at ease with regards to satisfying high demand for our applications. We will also examine how to set scaling limits to ensure that we always stay under budget for our deployments.

Why scaling matters

One of the common use cases for OpenShift Online is developing a backend for mobile applications. If you are not familiar with writing mobile applications, don't worry. Follow along with me, as this use case is applicable to other types of applications as well.

Imagine that you have spent thousands of hours developing a mobile application with a backend hosted on a private server. The application is a calorie counter that looks up food data and information via a REST API call. You developed the application using a backend, as it wasn't feasible to store all of the food information on each device. Having a backend that provides the data also allows you to update the database without requiring each user of the application to update to a new version.

Sales have been slow during the initial rollout of your application, so one backend server is more than adequate to handle all of the requests that users of your application are performing. You wake up one morning and find out that your application is listed as a featured application on the app store. Suddenly, the number of users of your application has exploded and your backend server is not able to keep up with the number of requests. Users of your application become frustrated and leave a negative review while also deleting the application because the experience was slow and unresponsive. At the most critical time for your application, your backend server could not handle the increased amount of traffic, and you weren't able to add additional servers fast enough to keep up with the increased traffic before your users became frustrated. You think to yourself that you should have just started with additional servers, but you didn't want to spend the additional money to maintain servers when one was enough to handle all of the current traffic. Enter OpenShift and automatic scaling.

Vertical and horizontal scaling

There are two types of scaling that IT people are familiar with: vertical scaling and horizontal scaling.

You can think of vertical scaling as providing a single huge monolithic server that has large amounts of RAM and CPUs on a single server. This huge server is typically the only server that handles requests for your application. If the server is not able to handle the number of incoming requests in a timely manner, the solution is to throw more memory or CPUs into the server. For instance, traditional relational databases typically reside on a single server that holds all of the data. Of course, this data can be replicated for failovers, but generally, all requests go through a single server.

Horizontal scaling, on the other hand, is a new way of approaching the scalability problems that we see today with web-based applications. Horizontal scaling means having lots of smaller-sized servers working together to handle all of the requests of a user instead of having a single large server. To illustrate the differences between vertical and horizontal scaling, take a look at the following diagram:

Vertical and horizontal scaling

An easy way to envision how horizontal scaling works is to imagine eating a bag of potato chips. While eating these chips, you accidentally drop one on the ground but decide not to pick it up. You then go about your business and leave the chip where it fell. Not long after dropping the chip on the ground, an army of ants approaches the chip. These ants break the chip into thousands of tiny pieces and each ant takes a single fragment of the chip back to their home. Instead of a single ant trying to take the whole chip back to the colony, they all worked together as a team toaccomplish the goal. This is how horizontal scaling works.

Using automatic scaling

One of the unique features of OpenShift Online is the ability to automatically scale your application based on the amount of traffic that your application is receiving. The platform monitors the number of concurrent HTTP requests and will add additional gears to your application if the demand is not being met. The platform will then monitor your gears and determine when it is safe to begin scaling back when demand is being met.

Note

At the time of writing this book, if you have more than 16 concurrent connections, the platform will add an additional gear. For the current configuration limit of the number of concurrent connections, see the official OpenShift Online documentation.

To understand how scaling works on OpenShift, let's examine what happens when you create an application that is not scaled. For instance, suppose that we created an application named notscaled that uses the PHP-5.4 runtime and a MySQL database. We will create this application with the following command:

$ rhc app create notscaled php-5.4 mysql-5.5

When issuing the preceding command, a single gear will be created for you on the OpenShift Online platform. This single gear contains both the PHP runtime and the MySQL database, as depicted in the following diagram:

Using automatic scaling

Creating a scaled application with the command line

By default, all applications that are created on the platform are not scaled, which means that a single gear contains all of the application code and cartridges. In order to create a scaled application, you need to pass the -s flag to an application when creating it. For example, to create a scalable application with the PHP-5.4 runtime and a MySQL database, we will enter the following command:

$ rhc app create scaledapp php-5.4 mysql-5.5 -s

When creating a scaled application on OpenShift, the gear creation process differs slightly in that a load balancer is added to the equation and the database resides on its own distinct gear. A good way to think of a scaled application is not a single gear but a gear group that belongs to the application. For example, given the preceding command, our application would look like the following diagram from a conceptual viewpoint:

Creating a scaled application with the command line

You can verify that MySQL and PHP are indeed running on separate gears by issuing the following command to view the details for your application:

$ rhc app show scaledapp

You will see the following output:

scaledapp @ http://scaledapp-packt.rhcloud.com/ (uuid: 5383e2c8500446265e000271)

--------------------------------------------------------------------------------

Domain: packt

Created: 5:56 PM

Gears: 2 (defaults to small)

Git URL: ssh://5383e2c8500446265e000271@scaledapp-packt.rhcloud.com/~/git/scaledapp.git/

SSH: 5383e2c8500446265e000271@scaledapp-packt.rhcloud.com

Deployment: auto (on git push)

mysql-5.5 (MySQL 5.5)

---------------------

Gears: 1 small

Connection URL: mysql://$OPENSHIFT_MYSQL_DB_HOST:$OPENSHIFT_MYSQL_DB_PORT/

Database Name: scaledapp

Password: 3D2Lq4R-4jgs

Username: adminFshVG51

haproxy-1.4 (Web Load Balancer)

-------------------------------

Gears: Located with php-5.4

php-5.4 (PHP 5.4)

-----------------

Scaling: x1 (minimum: 1, maximum: available) on small gears

In the preceding output, we can see that we are consuming two gears for our application: one for the PHP-5.4 runtime and an additional gear for the MySQL database. You might be wondering why the application isn't consuming three gears, one for each of the following:

· HAProxy

· PHP-5.4

· MySQL

This is because when creating a scaled application on OpenShift, HAProxy and the runtime are initially collocated on the same gear. These two items will remain on the same gear until three additional runtime gears are added. At this time, the PHP-5.4 runtime will be disabled on the head gear that is collocated with HAProxy, which enables HAProxy to perform better under increased load.

Creating a scaled application with the web console

Creating a scaled application with the web console is a very straightforward process. Simply log in to the web console by visiting http://www.openshift.com.

In order to create a scaled application with the web console, you will need to perform the following steps:

1. Authenticate to the OpenShift platform.

2. Click on the Add Application button at the bottom of the application dashboard.

3. Select to create a new application based on the PHP-5.4 runtime.

4. Name the application webscaled.

5. Scroll to the bottom of the screen and select to scale the application based on the web traffic.

The preceding steps are illustrated in the following screenshot:

Creating a scaled application with the web console

Click on the Create Application button to begin the gear-creation process. Once the application has been created, click on the link titled continue to the application overview page. Once you are on this screen, add the MySQL database by clicking on the link highlighted in the following screenshot:

Creating a scaled application with the web console

Once the MySQL database has been added to your application, you will see that the application is consuming two gears, as shown in the following screenshot:

Creating a scaled application with the web console

That's all there is to it. OpenShift Online will now automatically scale your application when extra capacity is needed. This is truly a set-it-and-forget-it type of mentality.

Using manual scaling

While automatic scaling is great for unplanned traffic spikes, manual scaling is a better alternative when you know that you are going to have an increased demand for your application. Let's explore this further using the same example that we used previously in this chapter where we had a mobile backend that is hosted on OpenShift Online. Suppose that we get an e-mail from Apple or Google letting us know that our application is going to be featured in three days. Given that we know about the potential for increased demand in advance, we would want to go ahead and manually add a couple of gears to our gear group in order to ensure that our customers have a great user experience.

The first thing we need to do to manually scale is disable automatic scaling for our application. We can achieve this using a marker, as discussed in Chapter 11, Using the Marker and Action Hook System. To create this marker, open a terminal window and enter the following command, ensuring that you are in the application's root directory:

$ touch .openshift/markers/disable_auto_scaling

$ git add .

$ git commit –am "Disable automatic scaling for our application"

$ git push

Once you have disabled automatic scaling, I have found that restarting the application will ensure that HAProxy does not automatically scale down the manually added gears:

$ rhc app stop -a scaledapp

$ rhc app start -a scaledapp

Now that automatic scaling has been disabled, we can manually add additional gears using the RHC command-line tools:

$ rhc app-scale-up -a scaledapp

Once you enter the preceding command, a new gear will be created on the OpenShift Online platform and will be added to your application gear group. Once the gear has been added, the application code will be copied from the head gear to the newly created gear using the rsync Linux utility. Once the application code has been copied over, the platform will add the new gear to the HAProxy load balancer. Once the command is complete, you will see the following output on your screen:

RESULT:

scaledapp scaled up

To verify that an additional gear has been added to your gear group, issue the following command:

$ rhc app show -a scaledapp

At this point, you should see that your application is using three gears, as shown in the following output:

scaledapp @ http://scaledapp-packt.rhcloud.com/ (uuid: 5383e2c8500446265e000271)

--------------------------------------------------------------------------------

Domain: packt

Created: 5:56 PM

Gears: 3 (defaults to small)

Git URL: ssh://5383e2c8500446265e000271@scaledapp-packt.rhcloud.com/~/git/scaledapp.git/

SSH: 5383e2c8500446265e000271@scaledapp-packt.rhcloud.com

Deployment: auto (on git push)

Go ahead and add a couple of additional gears to clearly understand how gear creation works. For example, if we had added five additional gears to our application, we would see the following output on the command line:

scaledapp @ http://scaledapp-packt.rhcloud.com/ (uuid: 5383e2c8500446265e000271)

--------------------------------------------------------------------------------

Domain: packt

Created: 5:56 PM

Gears: 7 (defaults to small)

Git URL: ssh://5383e2c8500446265e000271@scaledapp-packt.rhcloud.com/~/git/scaledapp.git/

SSH: 5383e2c8500446265e000271@scaledapp-packt.rhcloud.com

Deployment: auto (on git push)

php-5.4 (PHP 5.4)

-----------------

Scaling: x6 (minimum: 1, maximum: available) on small gears

You can also view this information by looking at your application in the web console, as shown in the following screenshot:

Using manual scaling

Setting scaling limits

Often, there are times when you want to have your production application never fall below a certain threshold of gears that serve the application content. This would be true if you have a popular application where you know that in order to meet capacity, you always need at least two gears that serve the traffic. Fortunately, OpenShift supports this capability by allowing the developer to set a minimum number of gears using both the command line and the web console. To illustrate this, let's set the minimum number of gears for an application with the following command:

$ rhc scale-cartridge php-5.4 --min 2 --max -1 -a scaledapp

The preceding command will configure our application, which is scaledapp, to always have a minimum of two PHP-5.4 web gears associated with it. You might notice that we set the maximum to -1. The -1 value configures the application to allow scale-up events until the maximum number of gears for the account has been reached. Once the command has been executed, you will see the following output on the terminal screen:

This operation will run until the application is at the minimum scale and may take several minutes.

Setting scale range for php-5.4 ... done

php-5.4 (PHP 5.4)

-----------------

Scaling: x2 (minimum: 2, maximum: available) on small gears

We can also set the minimum number of gears using the web console. Log in to the OpenShift website and browse the overview page for your application. For example, the scaledapp overview page looks like the following screenshot:

Setting scaling limits

Once you are on the application's overview page, click on the link under the Scales tab that will allow us to configure the minimum and maximum number of gears for the application. Change the minimum number to 4 and press the Save button.

Setting scaling limits

While having the ability to automatically or manually scale an application is a great feature, one must also consider the cost associated with an infrastructure that is scaled out. The OpenShift platform allows users to control the costs associated with scaling by configuring the maximum number of gears to scale to. For instance, let's pretend that we want our scaledapp application to have a minimum number of one gear but also only support the ability to scale up to two gears. This will enable us to control the costs associated with the application by never exceeding the two-gear limit for the PHP-5.4 runtime. To perform this action, we will enter the following command:

$ rhc scale-cartridge php-5.4 --min 1 --max 2 -a scaledapp

After entering the preceding command, you will see a confirmation message, as follows:

This operation will run until the application is at the minimum scale and may take several minutes.

Setting scale range for php-5.4 ... done

php-5.4 (PHP 5.4)

-----------------

Scaling: x2 (minimum: 1, maximum: 2) on small gears

This can also be performed on the web console using the same method that we discussed previously in this section by setting the values on the scaling dashboard, as shown in the following screenshot:

Setting scaling limits

Viewing the load-balancer information

When scaling applications on the OpenShift platform, regardless of using manual or automatic scaling, a load balancer is used to pass requests to the appropriate gear. The load balancer that is implemented as part of the system is the popular HAProxy software load balancer.

Note

HAProxy is a software load balancer that is commonly used to improve the performance of web applications and services by balancing incoming requests over several servers. The load balancer is written in the C programming language and is used by some of the most popular websites on the Internet.

As part of the HAProxy installation on the OpenShift gears, a web dashboard is provided so that you can view the number of gears and traffic associated with an application. This dashboard is available at http://appname- domain.rhcloud.com/haproxy-status/.

For example, if our application was called scalledapp and our domain was called packt, the URL for the HAProxy dashboard will be http://scaledapp-packt.rhcloud.com/haproxy-status/.

Open your favorite web browser and point to the HAProxy dashboard URL of your application. The dashboard should look like the following screenshot:

Viewing the load-balancer information

You can also access the HAProxy dashboard by clicking on the link next to the Web Load Balancer cartridge on the application's overview page on the web console:

Viewing the load-balancer information

Customizing the scaling algorithm

Often, there are times when the default scaling algorithm that ships as part of your OpenShift gear is not meeting your needs. Perhaps you want to scale your application based on the memory usage instead of the number of concurrent users. Fortunately, there is an opportunity to modify this behavior by implementing your own algorithm that will be inside of the action_hooks directory that we learned about as part of Chapter 11, Using the Marker and Action Hook System.

Note

The default scaling control file is located on GitHub and is thoroughly documented. For a better understanding of how the default scaling action works, consider reading the source code located at https://github.com/openshift/origin-server/blob/master/cartridges/openshift-origin-cartridge-haproxy/usr/bin/haproxy_ctld.rb.

To illustrate how to write a custom scaling control file that will allow our application to be scaled based on memory, let's create a new application called memscale using the PHP-5.4 runtime:

$ rhc app create memscale php-5.4 -s

Once the application has been created, switch to the .openshift/action_hooks directory and perform the following command:

$ wget https://www.openshift.com/sites/default/files/haproxy_ctld.rb_.txt

This will download a sample control file that contains the necessary information to be scaled based on memory usage. We need to change the name of the file in order to remove the _.txt extension:

$ mv haproxy_ctld.rb_.txt haproxy_ctld.rb

Once you have the file downloaded and the correct filename applied, add the changes to your repository and then commit and push the file to your OpenShift gear:

$ git add .

$ git commit -am "Adding custom scaling"

$ git push

The interesting bits of code in the control file are as follows:

# Scale up when any gear is using 400M or more memory.

mem_scale_up = 419430400

# Scale down when any gear is using 300M or less memory

mem_scale_down = 314572800

# min_gears - Once this number of gears is met, don't try to scale down any lower

min_gears = 2

gear_list['web'].each do |uuid, array|

mem_usage = `ssh -i ~/.openshift_ssh/id_rsa #{uuid}@#{array['dns']} 'oo-cgroup-read memory.memsw.usage_in_bytes'`.to_i

if mem_usage >= mem_scale_up

@log.error("memory usage (#{mem_usage}) on #{array['dns']} is above threshold(#{mem_scale_up}), adding new gear")

self.add_gear

end

end

In the preceding code sample, the application will automatically scale up when more than 400 MB of memory is used and will then scale down when any gear in the gear group consumes less that 300 MB of memory. The memory information is collected by establishing an SSH connection to each gear in the gear group, and then reading the memory information from the oo-cgroup-read command.

Summary

In this chapter, we learned how scaling works on the OpenShift platform. We started by learning how to use automatic scaling to ensure that your application can handle unexpected spikes in traffic. We then learned how to leverage manual scaling in order to proactively add capacity to your application when you know in advance that you will experience increased demand. After this, we learned how to control costs by configuring your application to support the maximum number of gears. Finally, we explored how to customize the scaling algorithm to scale based on memory usage instead of the number of concurrent HTTP requests.

In the next chapter, we are going to learn how to extend the OpenShift platform using the DIY cartridge. This cartridge will allow you to execute and run any binary on your OpenShift gear.