Implementing NetScaler VPX (2014)

Chapter 4. Compression and Caching

Compression and caching are two important features of an ADC, as they allow for optimization of a service without involving the backend servers and without having to do any configuration on the clients that are connecting. So, we are going to cover the following in this chapter:

· What is caching and compression?

· How do they work?

· How to configure compression and caching for web services?

Let us explore what caching and compression is and how they work as follows:

· Compression: This is the ability to reduce the number of bits within data. More specifically, it is the ability to use fewer bits than in the original data, as it is compressed.

· Caching: This allows for a NetScaler unit to store commonly accessed data in the RAM, which allows for faster fetching of data.

Both these features allow the client to get hold of the content faster, as it saves bandwidth between the service and the client. It can also reduce traffic to backend servers, and it can protect the backend servers from traffic storms. An important point to note is that these two features are not included by default in the Standard edition. In order to use these features, we either need to buy a feature license or upgrade to the Enterprise or Platinum edition. In order to use caching, we must upgrade to either NetScaler Platinum or NetScaler Enterprise and then buy a feature license.

So, let us start by taking a look at the compression feature of NetScaler.

Compression

The compression feature enables a NetScaler vServer to compress HTTP data that is going to or from the client. Another benefit of this feature is that the HTTP compression algorithm encrypts the data going from the client to the server and therefore adds another layer of security.

Note

Using compression does not encrypt the data as efficiently as using a digital signed certificate, so do not think of replacing certificates with compression.

This feature requires that the client who is requesting the content has a browser that supports compression. The newest and most common browsers, such as Firefox 4 and above, Google Chrome 20 and above, and Internet Explorer 7 and above, support HTTP compression. So, when a client connects to a vServer, it will announce what capabilities it has to the server. This allows NetScaler to choose the best type of algorithm.

Note

HTTP compression is based on the GZIP and DEFLATE algorithms. These are defined in the RFC 1950/1951/1952 formats. People interested in its technical aspects can read more at http://www.ietf.org/rfc/rfc1952.txt.

Now, the HTTP compression feature of NetScaler will compress data inside HTML, XML, CSS, text, and Microsoft Office documents. It does not compress any picture format files, JavaScript files, or other web files that are not text related. In order to configure compression in NetScaler, we first have to enable the feature globally in the appliance. This can be done using the following CLI command:

enable ns feature cmp

Here, cmp stands for compression. After we have enabled this feature in NetScaler, we have to activate it for a service. A service in this context can be a load-balanced service. This can be done using the following CLI command:

Set service nameofservice –CMP yes

This can also be done through the GUI under Traffic Management. Then, click on Service and go to the Advanced pane. Go to the Settings part of the window and enable Compression, as shown in the following screenshot:

Note

How to use Wireshark to analyze network traffic using filters and looking into different HTTP headers will be covered as part of Chapter 5, High Availability and Traffic Analysis.

Implementing compression policies

Now, after compression has been enabled, NetScaler will use the default policies that are set at a global level. We can see that, after we enable this for a service, it will automatically start compressing data for that service. If we go to the Compression Policy Manager window under HTTP Compression and click on Switch to Classic Syntax, then on Global, we can see the policies that are applied on a global level.

Note

The reason why we need to go for the classic syntax here is that, in the global settings of the compression feature, we have a configuration that defines which policies are processed and which are not. By default, this is configured to be policy type Classic. This is covered later in this chapter.

By default, there are five global policies. Each of the policies has an action attached to it; they are explained as follows:

· ns_nocmp_xml_ie: This policy does not compress when a request is sent from Internet Explorer. The content type is either text or XML.

· ns_nocmp_mozilla_47: This policy does not compress when a request is sent from Firefox. The content type is either text or XML.

· ns_cmp_mscss: This policy compresses the CSS file when the request is sent from Internet Explorer.

· ns_cmp_msapp: This policy compresses files that are generated by Microsoft Word, Excel, or PowerPoint.

· ns_cmp_content_type: This policy compresses data when the response contains text.

These policies can be viewed in the following screenshot:

What these policies do is that they do not compress data coming from the client to the services, but they will compress data that is generated from the servers, which contains either CSS files, Microsoft Office documents, or text.

After we have enabled compression for a service, we can test it by running a few HTTP requests against a service, for example, by opening a web browser to a service we defined in NetScaler. In my example, I have a simple IIS server setup, where I query the index page.

To view statistics, we can use the following CLI command:

Show cmp stats

We can also go through the GUI under HTTP Compression | Statistics, as shown in the following screenshot:

We can see that it has already managed to compress about 50 percent of the data. This feature uses the CPU of the appliance. So make sure that you do not enable compression if you have large amount of services, as NetScaler uses a large amount of CPU to perform compression.

Defining global compression settings

We can define some global settings to make sure that the compression feature does not run if NetScaler exceeds a particular CPU usage. Go to Optimization | HTTP Compression | Settings | Change Compression Settings. Here, we can define the followingparameters:

· Quantum Size: This parameter defines the amount of data that has to go through before NetScaler starts to compress data. The default value here is 57344 or 57 KB.

· Compression Level: Here, we can define at what level NetScaler should compress data. Best performance equals less compression and less CPU usage. Best compression equals more CPU usage.

· Minimum HTTP Response Size: This parameter defines what the minimum size of an HTTP response must be before it starts to compress. This should be set at a minimum of 100 KB so that NetScaler does not use a lot of CPU to compress small responses.

· Bypass Compression On CPU Usage: Here, we can define a percentage of CPU usage before NetScaler bypasses the compression feature. By default, this is set at 100 percent, which means that if NetScaler has 100 percent CPU usage, the compression feature is bypassed.

· Policy Type: Here, we can define a classic policy or an advanced policy. When we enable compression at a global level, it means that all classic policies will be enabled. If we create an advanced policy and bind it globally, it will not be processed if the policy type is classic.

· Allow Server-side Compression: If we have this feature enabled, it allows the backend web servers to enable compression. This feature makes NetScaler remove the Accept-Encoding header on all requests going to the web server, which makes the web server respond with an uncompressed response. This makes NetScaler respond back to the client with a compressed response.

· Compress Push Packet: This allows NetScaler to avoid waiting until it reaches the quantum size before it can start to compress data, when it receives a TCP packet with the PUSH flag enabled.

· Vary Header: This feature adds the Vary header to HTTP responses that are being compressed. This is mostly used in caching scenarios.

· External Cache: If private caching is enabled, NetScaler will add private cache-control to the HTTP header to make sure that the data is intended for a single user.

Now, most of these settings will be at their default values, but if you have a scenario where you, for example, have lots of large services and web servers with backend-enabled compression, you will need to change some settings here to make sure that it works properly.

Creating custom compression policies

We have gone through the different settings; it is now time to create our own compression policies. A policy is built up of a rule and an action. The rule can contain a query, for example, a client who is connecting using Firefox version 4.7. The default action for this rule would be to compress data.

1. Go to Optimization | HTTP Compression | Policies | Add. First, we need to change to classic syntax because of the global configuration of the policy type.

2. Next, add an expression in the Expression window. Note that we can choose from a list of predefined expressions.

3. As an example, we can choose General and HTTP client is Microsoft Internet Explorer, and add the action Compress.

4. The expression should be similar to the one shown in the following screenshot:

5. Click on Add Expression, and then on Create.

6. If we choose Add here, we can define a custom expression based upon the URL, header, IP, and so on. I'm not going to cover all the different types of expressions that we can use. Citrix has a good overview of the different expressions we can use in classic syntax at http://support.citrix.com/proddocs/topic/ns-main-appexpert-10-map/ns-pi-gen-expr-ref.html.

7. After we have created the policy, we have to either bind the policy to a service or bind it globally. Go to Compression Policy Manager | Switch to Classic Syntax | Global. Then, click on Insert Policy and choose the newly created policy.

Note

An important point to note is that classic and default syntaxes can perform the same type of evaluations, but the default syntax policies can also analyze deeper within the data, for example, the body of a HTTP request. It is therefore recommended to start using default syntax policies.

Testing our compression policies

If you wish to test a policy against a service, bind it to the service and define a low-numbered priority to make sure that it applies before other policies. In this example, we've added the newly created policy to the global level, and set it at priority 100 so that we can make sure that the policy is applied for all connections made from Internet Explorer. We can also unbind all the other policies to make sure that no other policies interfere with the one for Internet Explorer.

So, when we try to open a connection from Internet Explorer, we will be able to see from the packets that the traffic is compressed from the HTTP request header. We can see this in the following screenshot in the Content-Encoding field, which says it is compressed with gzip.

If we do the same for Google Chrome and analyze the traffic in Wireshark, we can see in the following screenshot that the traffic is not compressed and the data is sent in clear text, as there is no policy that involves an expression containing an action for Google Chrome:

We have now created a custom policy for Internet Explorer users and explored the different options for compression and how it works. It is time to continue on with the other optimization feature, that is, caching.

Caching

With the use of caching, we allow NetScaler to store frequently accessed objects in its cache area, which uses the RAM of the appliance as storage. Note that the MPX utilizes the SSD drive as the cache drive. This allows NetScaler to serve clients directly without requiring a trip to the backend servers. This offloads the backend servers and improves the overall performance of the services.

Now, caching is not something new; it is used everywhere. Most of the common web browsers have some sort of a local cache; many of the ISP vendors also have some sort of caching involved for frequently accessed services. Also, it is important to note that this feature requires a license in place, either a NetScaler Platinum license, or an add-on to Enterprise. Note that the caching feature is available only when using the HTTP protocol, and it can cache the following two types of data content:

· Static data: This includes CSS files, JavaScript files, images, static HTML pages, and so on.

· Dynamic data: This includes the dynamic catalog view, automatically generated files, and so on.

Enabling caching

Before we process any configuration, we first need to enable the caching feature. This can either be done via the GUI by right-clicking on Integrated Caching under Optimization and enabling this feature, or by using the following CLI command:

Enable ns feature IC

Before we do any other configuration, we also need to define some of the global settings for caching. This can be done under Optimization | Integrated Caching | Change Cache Settings. After we have enabled integrated caching, the default value for memory usage limit is set to 0, which means that NetScaler will not cache anything. Therefore, we need to put a value here. The maximum value that we can set is half of the memory available on the appliance. After we have put a value here, we can leave the rest of the settings at their default values and continue with the creation of a policy.

The caching feature consists of the following:

· A policy

· An action

· A content group

In the policy, we define an expression (rule) that needs to be evaluated, and then an action that defines what to do with the data defined in the rule, for example, to cache the objects or not. Then, we need to define a content group. This is where NetScaler is going to store the cached data for the objects.

Creating a content group

As an example, let us set up a basic caching policy that caches PDF files. Then, we are going to bind that policy to a load-balanced service.

First, we need to create a content group. This can be done by going to Optimization | Integrated Caching | Content | Groups and clicking on Add. Here, we enter the name of the content group, for example, PDF group, define an expiration timer (60 seconds), then go into the Memory pane, and define that NetScaler does not have to cache if the file is over 5,000 KB. By adding these parameters, we allow cached content to live in the content group for 60 seconds before it expires, and then be removed from the content group. We also define that NetScaler is not going to cache objects larger than 5 MB. We can leave the rest of the settings at their default values.

If you wish to know more about the different parameters in the content group, you can view the eDocs article at http://support.citrix.com/proddocs/topic/ns-optimization-10-map/ns-IC-setbasiccntgrp-tsk.html.

Creating a caching policy

Now that we have created a content group, we can create a policy. Go to Policies and click on Add. Now, we need to define an expression that only includes PDF files, attach an action, and configure where NetScaler is going to store the files. Give the policy a name, choose CACHE as the action, change Store in Group, and choose the content group that was created earlier.

In the Expressions field, type the expression HTTP.REQ.URL.ENDSWITH("pdf"). This expression allows NetScaler to cache all objects where the URL ends with pdf. We could also use the HTTP.REQ.URL.CONTAINS(\"pdf\") expression if we had pages containing links to PDF reports.

If you wish to combine multiple expressions in a policy, you can see the example at http://support.citrix.com/proddocs/topic/ns-optimization-10-map/ns-IC-configcacheobjectandstats-tsk.html. After we have added the expression, the window looks like the following screenshot:

Lastly, click on Create. Now that we have created the policy, we have to bind it to the vServer for which we want caching enabled. Go to the Integrated Caching menu and open Cache Policy Manager. Go to LB Virtual Server and choose the load-balanced vServer we wish to bind it to. Then, click on Insert Policy and choose the newly created policy. Click on Apply Changes and close the window.

We have now created a caching policy and attached it to a vServer, which has a PDF document available. When the first client connects to the vServer and tries to get the content, NetScaler will place it in the cache.

This can be viewed under Integrated Caching | Statistics, as shown in the following screenshot:

As it is the first connection, we will see a miss and one cached object. If we go to Integrated Caching | View Cache Objects, we can see what data is stored in the cache. We can also see that the PDF file is stored in the content group we defined earlier, as shown in the following screenshot:

As we can see from the statistics, when a new client connects, it will be served from the cache directly.

A new client will be allowed to get data from the cache as long as the content it is trying to access has not expired. If we open an object from within View Cache Objects, we can see how much time an object has before it gets expired.

After we have created a caching policy, there are some settings we can do in order to fine-tune the configuration and improve the performance of this feature.

Fine-tuning caching

One problem with caching is when multiple users try to access the same data simultaneously and NetScaler is not finished with downloading the object into the cache. This leaves the cache in a state where multiple users are directed to the backend servers instead of the cache.

There are some changes we need to make to ensure that clients do not bypass the cache. They are explained as follows:

· Prefetch: This allows NetScaler to refresh the content before the object expires.

· Flash Cache: When a large number of users try to access the same content, NetScaler sends only one connection to the backend server, and all subsequent requests are queued up. A single response is used to respond to all the users.

Both of these settings can be configured in the content group, which stores the cached objects. We can do this by opening a content group, and going to Others | Flash Crowd | Prefetch. Here, we can define both of these values within the group. Under Prefetch, we have to define how often NetScaler should evaluate the objects in the cache and check the expiration, and define how often it should refresh the cache. In the same pane, we can also see the active flash cache.

There are of course other configurations that we can set to improve the caching mechanism. By default, when we set up a cache policy, it will cache everything that is picked up by the rule.

If we have multiple load-balanced services and we have caching enabled, the cache space will fill up quickly. There are of course some web services that are more frequently accessed than others, so we do not want the less-accessed resources to fill up the cache space. We can attach a rule that defines the minimum number of requests to a server before the caching rule gets enabled. In this way we can make sure that the most active web services get better usage of the cache. This configuration can be set under Content Group | Memory | Do Not Cache, if the hits are less. The number here is again dependent on the amount of traffic going towards the different web services. If you are unsure about the amount of traffic going to the different vServers and services, you can use CLI commands.

For services, the following CLI command is used:

Stat service servicename

For vServers, the following CLI command is used:

Stat lb vserver vservername

This gives you a general idea of where to set the hit rate of objects that have been fetched from the cache.

Note

If you are unsure of the prefix on a CLI command, you can use the Tab key. For example, if you type stat and press the Tab key, you will get a list of the available options.

It is important to note that you can use the compression feature with the caching feature. This enables NetScaler to send compressed data from the cache to the clients that support compression.

If you want to see a graphical overview of the policies and their statistics for a vServer, you can check the visualizer feature. This can be done by right-clicking on a vServer in the NetScaler GUI and choosing Visualizer. This can be seen in the following screenshot:

Here, we can choose filters based upon what we want to display in the GUI. If we choose Caching and Compression from the top pane, we can see what policies are attached to the vServer. Also, we can view statistics for a policy by marking it. This is a good way to see what policies are actually attached to a vServer and what its statistics are.

Summary

We have now configured compression and caching for NetScaler. Both of these features provide a drastic improvement of web services. They might use more CPU and memory in NetScaler. So, you need to plan properly.

In the next chapter, we will go through how we can configure high availability, application firewall, and traffic analysis.