Hack and HHVM (2015)

Chapter 9. Configuring and Deploying HHVM

At the language level, HHVM is meant to be a drop-in replacement for the standard PHP interpreter. When running scripts from the command line, this promise generally holds. However, the way you configure and deploy it to serve web apps is different, not least because of its just-in-time (JIT) compiler.

In this chapter, you’ll learn the basics of setting up HHVM to serve web traffic. There are, of course, many details that depend on your specific application and infrastructure, so this chapter necessarily can’t be a complete guide. The aim is to give you a good enough understanding of HHVM that you can figure out how to integrate it with your setup.

This chapter doesn’t cover setting up the Hack typechecker, which is only used during development. For that, see Chapter 2.

Specifying Configuration Options

HHVM has a vast set of configuration options—far too many to cover them all in detail in this book. Many of them aren’t meant for end-users anyway; they’re for people hacking on HHVM itself. In this section, we’ll cover how to set configuration options, and what the most important ones are.

HHVM uses configuration files in INI format, which is the same format that the standard PHP interpreter uses. You can specify a configuration file with the -c flag:

$ hhvm -c config.ini file.php

INI format is very straightforward. Each option consists of a key-value pair. Each pair is on its own line in an INI file, with key and value separated by an equals sign. Whitespace is not significant.

hhvm.dump_bytecode = 1

hhvm.log.file = /tmp/hhvm.log

Some configuration options are associative arrays. hhvm.server_variables is an example; it sets the contents of the $_SERVER variable within PHP and Hack. With options like this, you specify them like this (example is an INI file):

hhvm.server_variables[ENVIRONMENT] = prod

hhvm.server_variables[A_NUMBER] = 314

Within a PHP or Hack program under this configuration, $_SERVER will have those values under those keys:

var_dump($_SERVER['ENVIRONMENT']); // Prints: string(4) "prod"

var_dump($_SERVER['A_NUMBER']); // Prints: int(314)

You can also specify options directly on the shell command line. Use the flag -d, followed by an INI-format key-value pair. Make sure that the pair either doesn’t have whitespace in it, or is quoted, otherwise the shell will split it into multiple arguments and HHVM will misinterpret it.

$ hhvm -d hhvm.dump_bytecode=1 file.php

You can combine multiple config files and direct options in the same command.

$ hhvm -c config1.ini -d hhvm.dump_bytecode=1 -c config2.ini file.php

In this way, the same option can be specified multiple times. HHVM reads the command line left-to-right, and config files top-to-bottom; the value of an option that it ends up with is the one that it reads last. For example, in the above command line, the option -d hhvm.dump_bytecode=1will override any setting of hhvm.dump_bytecode in config1.ini. If the option is also specified in config2.ini, that setting will win.

Generally, for production use, it’s best to specify all options in a single config file, simply to ensure consistency by avoiding this ordering dependence.

Important Options

hhvm.enable_obj_destruct_call (boolean, default off)

If this option is off, as it is by default, HHVM will not run __destruct() methods on objects that remain alive at the end of a request. (It will run __destruct() methods as normal at other times.) If your application can tolerate this, it can be a significant performance win: instead of having to traverse every array and object still alive at the end of a request, it can simply deallocate all of the request’s memory in one shot. If the option is on, HHVM will run all __destruct() methods as normal, at all times.

hhvm.hack.lang.look_for_typechecker (boolean, default on)

If this option is on, as it is by default, HHVM will refuse to run Hack files unless it can find a Hack typechecker server process that is covering that file. In production, you should turn this off, since you won’t be running the Hack typechecker except in development environments. It will be automatically turned off if you are in repo-authoritative mode (see Repo-Authoritative Mode).

hhvm.jit_enable_rename_function (boolean, default off)

If this option is off, as it is by default, using the builtin rename_function() will raise a fatal error. Knowing that functions will not be renamed allows for some powerful optimizations, so if you don’t rely on this functionality, you should keep this option off.

hhvm.server.thread_count (integer, defaults to twice the number of CPU cores)

The number of worker threads that are used to serve web requests in server mode (see Server Mode). There’s no one-size-fits-all formula for the ideal thread count. It depends, in complex ways, on the application’s performance characteristics, and on the machine’s CPU and memory specs.

Apps that don’t use async (see Chapter 7) will likely benefit from thread counts much higher than the default, since threads will spend a good amount of wall time idle, waiting for I/O. A good starting point might be 15 times the core count.

Apps that uses async heavily (see Chapter 7) are likely to be OK with the default thread count, or slightly higher. Async can help HHVM use the CPU during time that would otherwise be spent idle.

The best way to tune this value is to experiment. Vary the thread count and observe the effects on CPU and memory utilization. As you raise the thread count, utilization of both resources should increase. Try doing this experiment at peak traffic times, since utilization at peak is the most important determiner of total capacity. Use thread count to raise utilization up to some defined limit (70% CPU, say), and stop there.

HHVM uses OS-level threads, unlike PHP-FPM, which uses processes. The overhead of increasing HHVM’s thread count is quite low, so don’t worry about that when increasing it.

hhvm.source_root (string, defaults to working directory of HHVM process)

This option is only relevant in server mode, where it holds the path to the root directory of the code being served.

Server Mode

HHVM has two primary modes: command-line mode and server mode. Command-line mode is what’s used when you run a command like hhvm test.php; it immediately executes the given script and exits when the script terminates.

Server mode is what you’ll use to serve web requests. In this mode, the HHVM process starts up and doesn’t execute anything immediately. It executes code in response to requests that come in via FastCGI, and stays running after requests finish. It can process multiple requests simultaneously. Just-in-time-compiled code is kept in memory (in the translation cache) and shared across requests.

Start HHVM in server mode with the command-line flag -m server:

$ hhvm -m server -d hhvm.server.type=fastcgi -d hhvm.server.port=9000

This is a FastCGI server, so the port number 9000 is conventional.

The next step is to configure a web server to send requests to the HHVM FastCGI server. You can use any FastCGI-compatible web server software, such as Apache or nginx. We’ll focus on nginx here, since it’s simpler to configure. We won’t learn how to configure nginx from the ground up,

The bare minimum for sending FastCGI requests to HHVM is a location directive like this:

location ~ \.(hh|php)$ {

include fastcgi_params;

fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

fastcgi_pass 127.0.0.1:9000;

}

fastcgi_params refers to a configuration file that comes with the standard nginx installation; it passes parameters of the request (HTTP method, content length, etc.) on to HHVM. The fastcgi_param directive tells HHVM which file to execute. The fastcgi_pass directive simply means that the request will be passed to the FastCGI server at the address 127.0.0.1 on port 9000. This configuration will be applied to any request for a path ending in .hh or .php.

THE WRAPPER SCRIPT

HHVM’s command line interface can be quite complex, so the package includes a script called hhvm_wrapper that puts a more convenient interface on some of the more common options.

Run hhvm_wrapper --help to see the options it provides. It can do things like run a script in repo-authoritative mode (see Repo-Authoritative Mode) with a single command:

$ hhvm_wrapper --compile test.php

To see what’s going on behind the scenes, add the flag --print-command to any hhvm_wrapper command line. It will print out the underlying HHVM invocation, instead of running it.

Warming Up the JIT

The first few requests to an HHVM server will be slower than the rest, because it has to compile the PHP and Hack code into machine code before executing it. The effect is noticeable enough that you shouldn’t immediately expose a newly-started HHVM server to production traffic; you should warm it up first, by sending it a few synthetic requests.

In fact, the server starts out by not compiling code at all. The first few requests are run in HHVM’s bytecode interpreter. The theory is that the first few requests to a web server are unusual—initialization is happening, caches are being filled, etc.—and that compiling those codepaths is bad for overall performance, since they won’t be frequently taken once the server is warmed up. HHVM also uses these requests to collect some profiling data about the data types it sees the code using, so it can compile more effectively later. You can tune this threshold with the optionhhvm.jit_profile_interp_requests.

TIP

This characteristic sometimes confuses people trying to benchmark HHVM. They see the first few requests being slower than they expect—in fact, often slower than running the same script from the command line. (HHVM always compiles code when running command-line scripts.)

To benchmark a JIT-based execution engine properly, you have to give it a warmup period. Because doing so is somewhat subtle, the HHVM team has released a tool that does a lot of it for you, enabling consistent benchmarking of server workloads. It’s available on GitHub.

Sending warmup requests can be as simple as using the command-line curl utility, from a shell script or similar. For best results:

§ Use requests that are a representative mix of the most common requests you expect to see in production. For example, if you expect that about 40% of all requests in production go to index.php, then about 40% of your warmup requests should be to index.php.

§ Avoid sending multiple warmup requests in parallel. Nothing will break if you do send multiple parallel requests, but the JIT compiler tends to generate better code if it is not working on multiple requests at the same time.

Eventually you should have the warmup process scripted so that you can warm up a server with a single command, but initially, you’ll need to have some manual involvement. There’s some subtlety to working out a good number of requests to send—it varies depending on your application.

One good way to work out a good number is to keep sending requests until you’re seeing consistent response times for requests to the same endpoint. Even after the JIT compiler kicks in after the eleventh request, it will go through another jump in performance later on, as it starts recompiling some code with the benefit of profile-guided optimization (PGO). There isn’t a single request-count threshold for when PGO begins—it’s based on how frequently individual pieces of code are run—so you should keep running warmup until response times level off.

You can also use the admin server (see The Admin Server) to monitor the sizes of the compiled-code caches. They will grow rapidly when the JIT compiler starts, but their growth will soon slow down significantly. When that happens, the server is sufficiently warmed up.

Repo-Authoritative Mode

By default, HHVM is continually checking your PHP and Hack source files to make sure they haven’t been modified since it last read them. When deployed to production, this can incur significant costs for no benefit, since source files aren’t actually changing frequently in production.

To fix this, HHVM offers repo-authoritative mode. In this mode, you build a bytecode file (the repo) from your codebase ahead of time, and deploy that to production, without source files. Then, the HHVM server process reads from the repo and never checks the filesystem for source files—that is, the repo is “authoritative”.

As well as reducing the need for filesystem operations, repo-authoritative mode allows HHVM’s compiler to make significant optimizations that it otherwise can’t. Because of the guarantee that the compiler can see all the code that can possibly exist in the process’ lifetime, it can do things like function inlining that aren’t normally possible.

The inability to introduce new code at runtime means that in repo-authoritative mode, you can’t use eval(), and you can’t use create_function() (which is actually just a wrapper around eval()).

Building the Repo

Before deploying, you have to build the repo. You do so by passing several flags to HHVM. We’ll start with the most basic example: building a repo that contains a single file.

$ hhvm --hphp -t hhbc -v AllVolatile=true test.php

The --hphp flag^[34^] signals that we want to do some offline operation, instead of executing PHP or Hack code. -t hhbc means “target HHBC”—that is, we want to output bytecode. -v AllVolatile=true turns on an option that disables a rather aggressive optimization that takes some care to use correctly^[35^]. Finally, we pass filenames to produce bytecode for—in this case, only one.

This results in a file named hhvm.hhbc in the current working directory; this is the repo. The repo is actually just a SQLite3 database file, so you can use the sqlite3 command-line tool to examine it^[36^].

In practice, it may be awkward to name all the source files that need to be included on the command line, so HHVM can also accept the name of a file that contains one source file name per line. Let’s say we have a file files.txt:

lib/a.php

lib/b.php

index.php

Then we can tell HHVM to use this list as its input:

$ hhvm --hphp -t hhbc --input-list files.txt

The repo captures the file paths that were passed to HHVM when building the repo, and these paths form a “virtual filesystem”, of sorts, for the HHVM process that runs from the repo. In concrete terms, when HHVM is running from this repo and the web server gives it a request for some path—say /some/file.php—HHVM will look in the repo for a file that was at the path some/file.php when the repo was built.

In view of that, things will generally be easiest if you build the repo from the path that will correspond to your web server’s document root when deployed. This may vary depending on how much path rewriting you intend to do at the web server level.

Deploying the Repo

Copy the hhvm.hhbc file to your production servers. You don’t need to copy source files; HHVM will run without them. If you have your web server configured to look in the filesystem to determine what to do with a given request (which is fairly common), you may need to copy your source files for that purpose. However, you could instead just have empty files in a directory structure mirroring your actual codebase, and that would be enough for the web server.

You must use the same HHVM binary to run from the repo as you do to build the repo. Repos are not backward-compatible or forward-compatible. When building a repo, HHVM embeds in it a repo schema ID, unique to each HHVM version. When using a repo at runtime, HHVM checks the repo’s schema ID against its own, and won’t use the repo if the schema IDs don’t match.

It doesn’t matter where you put the repo file, as long as the HHVM process can read it. Remember the “virtual filesystem” formulation from before—in repo-authoritative mode, HHVM will never be concerned with the real filesystem, only with the contents of the repo. This also means that the setting of hhvm.server.source_root is irrelevant in repo-authoritative mode.

There are two relevant configuration options: one to tell HHVM to use repo-authoritative mode, and one to tell it where the repo is. This is what you would put in your INI file:

hhvm.repo.authoritative = 1

hhvm.repo.central.path = /path/to/hhvm.hhbc

The Admin Server

In server and daemon modes, HHVM provides a mechanism by which you can inspect and control the running server process. The process listens on a separate port, over which you can issue commands with HTTP requests. This functionality is called the admin server. It offers a wide range of commands, and we won’t look at all of them in great detail here.

The admin server is turned off by default. You turn it on by specifying a port number for it to listen on—the specific port doesn’t matter, as long as it’s free and HHVM can bind to it. You should always specify a password as well. You’ll give the password along with every request to the admin server.

hhvm.admin_server.port = 9001

hhvm.admin_server.password = 9UejLK2jVhy

You’ll need to set up your web server to listen on a new port and forward requests to the admin server’s port as FastCGI requests. In the examples ahead, we’ll assume that we picked port number 15213 for the web server to listen on.

WARNING

The admin server is potentially very dangerous, which is why it’s not enabled by default. Never enable it without a password—a strong password, which you rotate regularly—and don’t expose its port to the Internet.

Once the HHVM server process is started, you can use the curl command-line utility (or a web browser) to send commands to it. In these examples, we’ll assume that we’re running curl on the same machine as the HHVM server process, and that the admin server password is as configured above.

Making a request to / on the admin server will show a help message with a list of possible commands. You don’t need to provide the password to get help.

$ curl http://localhost:15213/

/stop: stop the web server

instance-id optional, if specified, instance ID has to match

/translate: translate hex encoded stacktrace in 'stack' param

stack required, stack trace to translate

build-id optional, if specified, build ID has to match

bare optional, whether to display frame ordinates

/build-id: returns build id that's passed in from command line

/instance-id: instance id that's passed in from command line

/compiler-id: returns the compiler id that built this app

/repo-schema: return the repo schema id used by this app

/check-load: how many threads are actively handling requests

/check-queued: how many http requests are queued waiting to be

handled

/check-health: return json containing basic load/usage stats

/check-ev: how many http requests are active by libevent

/check-pl-load: how many pagelet threads are actively handling

requests

/check-pl-queued: how many pagelet requests are queued waiting to

be handled

/check-mem: report memory quick statistics in log file

/check-sql: report SQL table statistics

/check-sat how many satellite threads are actively handling

requests and queued waiting to be handled

... more items omitted ...

Each of the commands is a path that you can add to your admin server request. For any of these commands, you have to provide the password that you configured the admin server with, under the GET parameter auth. For example, here’s the command compiler-id, which shows the Git revision that this HHVM binary was built from.

$ curl http://localhost:15213/compiler-id?auth=9UejLK2jVhy

tags/HHVM-3.6.2-0-g11e5cecb678453d47ce2cea83997a2c5703abb41

Indented items in the help, like instance-id under /stop, are the names of GET parameters that you can provide to the command:

$ curl http://localhost:15213/stop?auth=9UejLK2jVhy&instance-id=INSTANCEID

If you don’t provide the right password, the admin server returns the text Unauthorized.