eCommerce in the Cloud (2014)

Part II. The Rise of Cloud Computing

Chapter 6. Virtualization in the Cloud

Virtualization is a key enabler of cloud computing but it is not part of the cloud itself. As previously discussed, the cloud is defined by the following three attributes: elastic, on demand, and metered. Virtualization itself isn’t one of those three attributes but it does help to enable all three.

NOTE

Virtualization is a key enabler of the cloud but it is not the cloud itself.

Virtualization in its various forms enables the following:

Partitioning

Vendors don’t make any money if they can’t pack a lot of their customers on each physical server. Many vendors offer virtualized servers with as little as one physical hardware thread. There are typically two hardware threads per physical CPU core, 12 cores per processor, and two processors per physical host. That means a single commodity machine can be divided into 48 servers, each capable of being rented out to an individual or organization. Density is how vendors make their money.

Isolation

Given that partitioning hardware is required to make business models work, isolation is required to keep individual partitions from interfering with each other. You wouldn’t want a CPU-intensive workload like DNA sequencing to interfere with your application server’s ability to respond to HTTP requests. You can provision whole servers from many Infrastructure-as-a-Service vendors, just as you would secure fractional slices of physical servers.

Portability

Virtualization has tooling that makes it easy to take snapshots of live running systems. These snapshots, as we discussed in the last chapter, can be used as the starting point when provisioning new hardware. Everything is fully encapsulated in these snapshots, making them fully portable across other servers within a cloud or even across clouds.

Workload shifting

With virtualization serving as an abstraction layer over the hardware, some cloud vendors offer the ability to move live running virtual machines from one physical host to another. Sometimes your cloud vendor even does this for you automatically while performing maintenance or as the hardware is experiencing a fault of some sort.

Isolation and portability benefit ecommerce most. Isolating workloads on shared hardware is a necessity, given the sensitive nature of the data involved with ecommerce. Portability is a key enabler of building deployment units very quickly, as you can work with predeveloped snapshots instead of having to build up each system from scratch.

While virtualization has benefits, it also has drawbacks that have wide-ranging implications for your deployment architecture. Let’s review the different forms of virtualization and the implications of each.

What Is Virtualization?

Virtualization is more of a concept than an actual implementation of any given technology. Virtualization offers you the ability to partition a physical server into many smaller servers, with each virtual server isolated from other virtual servers on the same physical server. The technology behind virtualization has been around since the beginning of the mainframe days. At its core, there’s typically an abstraction layer called a hypervisor that is responsible for splitting up a system’s resources and making them available to guests.

There are three broad approaches to virtualization, which we’ll cover in the following sections.

Full Virtualization

In full virtualization, the hypervisor is installed directly on bare metal. The hypervisor allocates the server’s physical resources (CPU, memory, disk, NIC) to virtualized servers. Operating systems are installed directly on the virtual servers, with the physical resources allocated to each virtual server being the only resources each virtual server has access to.

The advantage of this approach is that you don’t have to make any changes to the operating system. You can install any operating system the underlying hardware supports and not have to make any changes. Each operating system is entirely oblivious to the other operating systems on the same physical server. This type of hypervisor lends itself well to public Infrastructure-as-a-Service clouds. Examples of full virtualization include Xen Hardware Virtual Machine, Linux KVM, and Microsoft Hyper-V. Figure 6-1 shows how full virtualization carves up each physical machine.

Figure 6-1. Full virtualization

Full virtualization is best when you need to support many different operating systems, especially those that do not support a limited form of virtualization known as paravirtualization. Most clouds support both full virtualization and paravirtualization.

Full virtualization suffers from performance degradation because all physical resources must be accessed through the hypervisor. The performance of full virtualization is starting to be improved through the use of hardware virtual machine (HVM) extensions to x86 processors that make it easier for the guest operating system to bypass the hypervisor and use the CPU directly. But paravirtualization takes things a step further and allows more native access to many resources.

Paravirtualization (Operating System–Assisted Virtualization)

Traditional full virtualization requires that every system call be trapped by the hypervisor and passed back to the physical host. Depending on the workload, performance can be dramatically worse than if the operating system was not running on top of a hypervisor. In _paravirtualization, the kernel works cooperatively with the hypervisor to pass through certain calls directly to the underlying hardware. Calls such as those for memory management, time keeping, and interrupt handling are often passed right through from the guest to the host.

TIP

While the term paravirtualization may be new to many readers, the concepts and even much of the technology have been in use for decades. Almost all operating systems bypass the hypervisor in some way.

Paravirtualization, shown in Figure 6-2, requires that the kernel be substantially modified so it can work with the hypervisor. Most modern Linux kernels, along with some Unix kernels, support paravirtualization, so you probably won’t ever have to do any of this work on your own.

Figure 6-2. Paravirtualization

Paravirtualization is great because you get isolation offered by full hardware virtualization, yet you get nearly the performance of operating system–level virtualization. So long as your Infrastructure-as-a-Service and kernel vendors support it, paravirtualization is the best approach. They incur the costs of making the kernel and hypervisor work together, and you reap the rewards of better performance than full virtualization with nearly no overhead. Common implementations include Xen and VMware.

Paravirtualization is now the most prevalent virtualization option offered by public Infrastructure-as-a-Service vendors.

Operating System Virtualization

In operating system virtualization, a very light hypervisor is installed on the operating system itself (see Figure 6-3). With this approach, guest operating systems share the kernel of the parent operating system. The kernel offers each guest operating system its own caged sandbox so that processes cannot interfere with each other. Some forms of operating system virtualization offer resource throttling so a given guest cannot consume more than a specified amount of system resources. This approach is entirely dependent on what the operating system you’re using supports.

This technology has been around since the days of mainframes and continues to be used today. Today’s well-known incarnations include FreeBSD Jails, Solaris Containers/Zones, and Linux Containers (LXC).

Figure 6-3. Operating system virtualization

The following sidebar contains a humorous explanation of operating system virtualization.

LINUX CONTAINERS (LXC), EXPLAINED

Think of every process on the machine as being like a five-year-old. If you have a bunch of them, they’re eventually going to start fighting. Ever seen five-year-olds fight? It’s stupid:

“He’s standing near me, and I don’t like it!”

“I don’t want her to make that noise!”

In the context of a process on Linux, you can think of this as two server processes both wanting to listen on the same port, or writing a config file to the same place. And here you are, thinking, “Grow up and share. If you can’t handle this, wait till your first interaction with the DMV.”

But they will never share, because they’re five years old. They’ll just whine at each other to the point where you have to physically separate them just to preserve your own sanity.

One way to physically separate a dozen five-year-olds is to build a dozen houses, and give them each a house. This is like running a separate physical machine for each process. Yes, it solves the problem, but it’s expensive and wastes resources.

So then you try a virtual machine within one physical host. If we’re talking about separating five-year-olds, this is like turning one house into a bunch of apartments, one for each five-year-old. This allows for maximum configurability. Each five-year-old can set the thermostat at whatever temperature is comfortable. Sure, they don’t have as much space as they would if they had the whole house, but at least they’re contained. They’re not really aware of each other, but generally, they’re happy.

But if you’re Mom and Dad, this is just silly. Do you really need one separate kitchen for each five-year-old? No, that’s overkill. One kitchen is enough. Do five-year-olds really need to set the thermostat as they please? No, they’re five. Plus, I’m the parent, I pay the heating bill, and the thermostat isn’t going over 57 degrees all winter.

So what’s the solution?

You give each kid a magic pair of glasses that let them see everything but each other. They’re all walking around the house, aware only of their own existence. Ah, but what if they both want to play with the same toy? Simple, buy the same toy for each kid. A lot easier than building a house for each kid. In reality, they’re going to conflict over only a very small number of things in the house, so you can replicate those things, and the five-year-olds will be none the wiser.

This is what LXC is. It lets processes think they have complete run of the machine, but a very lightweight hypervisor keeps them separate. They’re all still running in the same kernel, but they don’t know it. An LXC container is a collection of files that represents the minimum set of “toys” the processes might fight over. It’s a bit more advanced, though. You can, for example, run a Red Hat LXC container inside an Ubuntu kernel. As long as your processes don’t have explicit expectations about the kernel, you’re fine. And most of the time, processes don’t have such expectations.^[48^]

This approach is becoming increasingly popular because of the performance it offers relative to running an operating system natively on physical hardware, while still providing enough isolation between tenants. With operating system virtualization, you have full native access to the CPU, memory, disk, NIC, and so forth, because you’re relying on a shared kernel that itself is natively accessing the physical resources. With full virtualization, you have to go through your guest kernel and then through a hypervisor before you can access any physical resources. There is potentially a lot of overhead when every operation must pass through the hypervisor.

With operating system virtualization, depending on how you configure it, you could allow the guests to cooperatively share resources, thus enabling higher density.

Platform-as-a-Service vendors have embraced operating system virtualization because they have no need to support a wide range of operating systems. These vendors tend to pick one operating system and standardize on it. You can pack many guests on a single physical host, because the overhead of each guest is so small. The performance is also much better than with some of the heavier virtualization techniques, while allowing for varying levels of isolation between guests.

Summary of Virtualization Approaches

Virtualization of various forms will continue to be used by cloud vendors for the foreseeable future. The vendors need to be able to pack multiple guests on a single physical host to make their business model work. At the same time, cloud users need to have isolation between their workloads and other workloads running on the same physical host.

The trend has been away from full isolation, in which everything is virtualized to lighter-weight solutions like paravirtualization, which is now the standard for most Infrastructure-as-a-Service vendors. Paravirtualization is a great trade-off between full hardware virtualization and operating system virtualization. It is also supported by most Linux kernels, so it’s fairly easy to deploy now. Most vendors offer two or three virtualization solutions, so you get to pick which one works best for you.

Table 6-1 shows a quick review of the approaches to virtualization.

Table 6-1. Review of approaches to virtualization

These approaches to virtualization are increasingly blending together. You have to research your vendor’s implementation of each of these approaches to find out exactly what, for example, paravirtualization means to your vendor.

Improving the Performance of Software Executed on a Hypervisor

The difference between the three high-level approaches come down to how many resources must be accessed through the hypervisor. With full virtualization, everything must be accessed through the hypervisor. With operating system virtualization, nothing passes through the hypervisor.Table 6-2 shows the three approaches to virtualization and whether each resource must be accessed through the hypervisor.

Table 6-2. Must access physical resources through the hypervisor

Full virtualization and paravirtualization are likely to suffer from some performance degradation because access to some physical resources must pass through a hypervisor. That will always take longer than natively accessing those resources.

As we discussed in Chapter 3, the single biggest hindrance to server-side performance is the number of calls to remote systems. It’s increasingly common to call out to multiple remote systems to assemble a single page. Figure 6-4 shows the systems most shopping cart pages have to call out to before they can be rendered.

Figure 6-4. Services often required to render a cart page

There are likely to be a number of calls out to each one of these remote systems, with each call being synchronous. As we discussed in Chapter 3, each call out to a third party requires that the potentially dozens or hundreds of packets representing the HTTP request and HTTP response travel over the path shown in Figure 6-5.

Figure 6-5. Intermediary required for host-to-host communication

A single page may require 50 of these synchronous calls. The real performance problem comes not from the time it takes for each system to respond to these calls but generally from the overhead associated with going back and forth.

TIP

Performance problems in ecommerce are almost always due to making too many calls rather than the response time of any given call.

Overhead comes in two forms:

§ Network latency

§ Latency incurred on each physical server

Virtualization is in large part responsible for the latency incurred on each server. Traversing TCP/IP stacks on each guest operating system turns out to be much slower if the guest operating system has to pass through a hypervisor to access the CPU, memory, and NIC. It’s best to bypass the hypervisor as much as possible, provided there is adequate separation between virtual servers. If you have to use a full hypervisor for some reason, try to bypass it for anything related to HTTP request handling through techniques like single-root I/O virtualization (SR-IOV). SR-IOV, which is now supported by many cloud vendors, allows your NIC to be presented to your guest virtual servers as if it was a physical device. Any calls from your virtual server to your NIC bypasses the hypervisor entirely.

Summary

In this chapter, we reviewed how virtualization is an enabler of cloud computing, but is not a defining attribute. We then reviewed the three basic approaches to virtualization and covered techniques used to improve performance. Next, we’ll discuss Content Delivery Networks.