Developing Web Components (2015)
Part IV. Testing, Building, and Deploying Components with Polymer
Chapter 16. Testing Web Components
Jarrod Overson
The biggest problem with any new technology is fitting it into an existing flow with minimal turbulence. The pieces of the flow that are subject to the most friction are:
Development (IDE assistance, documentation)
Where do you go for questions? How will you be able to solve problems that come up?
Maintenance
How stable is the technology? Is it undergoing massive changes? Will going back to code written with this tech one year from now be simple or a nightmare?
Testing
How are unit tests written? How testable is code written with this new technology?
Building
Does this new technology affect the build process at all?
Deploying
Do your deployments change? What metrics need to be tracked in order to measure the impact of the new technology?
It’s important to run through at least these checkpoints in order to get a high-level overview of the problems you might face, although it might be tempting to charge forward once you’ve gotten past the first hurdle because that’s usually the primary focus of technology marketing.
THE IMPACT OF MARKETING
By “marketing,” we mean whatever influences what frameworks and libraries become popular. It’s not simply a matter of advertising dollars spent by open source developers (though that can be a factor); it’s GitHub stars, Twitter followers, blog posts, documentation, websites, podcasts—the lot of it and more.
It’s all tailored to you, and it’s important to see through the viral fluff to find the answers to the questions you really have. Libraries, specs, frameworks, and tools are written by developers for developers, so they’re optimized for productive development.
Deploying Angular code almost seemed like it was an afterthought by the Angular team, as the community found it couldn’t be effectively minimized by current tools, affecting deployments and builds late in the process.
Backbone was the bee’s knees until many people found sparse documentation on building large applications, which affected long-term maintainability.
Web components are being touted as the future of everything (by this book’s authors, no less!) without having much real-world usage yet. The deficiencies are being filled in, but it’s still turbulence that will affect your flow. Make sure you look past our fluff and promises and find out how web components will really work in your flow.
The biggest problem with web components, in my humble opinion, is the lack of testability and the complications that web components introduce when testing, especially at this phase. It might seem silly; at the end of the day we’re just talking about HTML elements, which have been around since the middle ages (approximately). That may be true, but with the substantial polyfills that are used in different combinations for every browser on the market right now, the need for automated cross-browser testing is critical, and the tools that normally help us with that task aren’t yet perfectly suited for web component testing.
There are a number of different ways to test browser-destined code, each of which has its drawbacks when testing web components (in their current state) and web component polyfills. Here we’ll look at a few of the popular testing tools for web applications and see how they stack up.
PhantomJS 1
PhantomJS, as of version 1.9, is 100% out of the equation when testing web components due to its lack of support for the core technologies and the inability to work with some of the polyfills the Polymer group provides. The author, Ariya Hidayat, is at work on version 2.0, but as of yet it’s unknown how well PhantomJS will support web components in the future. This immediately kills a lot of common tasks and tools that leverage PhantomJS for headless testing and will likely affect a lot of people’s testing in Continuous Integration (CI) environments.
PhantomJS 2
By the time you read this book, PhantomJS 2 will have been released. It provides a capable headless browser with which to test shimmed web components. QTWebKit, upon which PhantomJS is based, does not have native support for web component technologies, but the shims go a long way toward providing a headless solution to unit testing web components. Polymer functions and behaves well in the environment and web components are of interested to the PhantomJS team, so the current level of support can be expected to be maintained or improved.
Selenium WebDriver
Selenium WebDriver is, by and large, the most commonly used tool for automating web page UI testing, and it is almost completely broken as soon as you include platform.js on your page (as of version 0.2.4). This is noted and the Google team is aware of it, but it means that if you maintain a large suite of automated frontend tests, they may all be rendered useless by the simple introduction of a single Polymer component—regardless of whether that component is even being tested.
These are horror stories that, if told around a campfire in the woods, would send some developers running for their sleeping bags, praying for morning to come and the nightmares to stop.
It’s not all fright and fear, though, and given the progress everyone is making, some of these issues may be rectified shortly after this book is published. Don’t take anything in this book as gospel (until the second or third revision, at least). These topics are discussed because they hold true at the moment and are important to be aware of.
Karma
Fortunately, one of the best modern day choices in testing browser code, Karma (formerly Testacular), is still a top-of-the-line choice for unit testing Polymer components. There are some quirks and a few edge cases that aren’t well supported, but it’s still a solid option that will probably expand to fill the role better in the future.
If you’re already familiar with Karma, you’ll probably still want to have a separate configuration in order to manage maintainability of the test framework going forward. It is highly likely that the practices outlined here will evolve and the plugins will change, so you don’t want the Polymer testing to get in the way of established and stable testing elsewhere.
For those just getting started with Karma, it’s a Node.js tool that can be installed globally with its per-project plugins installed locally as developer dependencies. You can install Karma via npm as follows:
$ npm install -g karma-cli
This will give you the tool that then delegates to a local installation of the Karma libraries. The base library you’ll need for Karma is, appropriately, karma:
$ npm install karma
Karma requires a sometimes extensive configuration, but thankfully, the boilerplate and some sensible defaults can be automatically generated via the init command from the Karma CLI. This can only be done after installing a local version of Karma, so don’t skip the previous step before running this command:
$ karma init
This will guide you through a series of questions that will help you to get started with most common testing implementations. We’re going to be working with Mocha and Chai, and you can choose the browsers you want to run when you get to the prompts. It’s recommended that you run at least one other browser outside of Chrome in order to ensure that you are using as many of the polyfills as possible. Chrome has full support for web components from version 36 onward, so the behavior and performance could be substantially different.
Outside of what is automated with init, a good set of modules to start with is:
karma
5H3 base Karma library
karma-mocha
A plugin that wires Mocha into Karma
karma-chai
A plugin for the Chai assertion library
karma-chrome-launcher
A launcher for Chrome
karma-firefox-launcher
A launcher for Firefox
karma-webcomponent-helpers
Some helper scripts to assist with loading fixtures for web component testing
These can be installed in one fell swoop via npm:
$ npm install --save-dev karma karma-mocha karma-chai \
karma-chrome-launcher karma-firefox-launcher \
karma-webcomponent-helpers
Part of the complication of using Polymer as our web component framework is that we are writing and storing all of our base component files as HTML and importing them as relative HTML imports. This is 100% foreign to almost every existing use case, and no testing framework supports this at all yet. Some may support loading fixtures, which can be used depending on the implementation, but this is largely a problem that still needs to be solved.
A unit testing framework specific to Polymer is something that could have potential, as could a unit testing framework for whatever web component manager ends up leading the pack in the next 12 months. There are not yet any true best practices for testing web components or Polymer, so take our recommendations with a grain of salt. They have worked, and currently do work, but as and when something new comes along, it should be evaluated with preference.
The primary tasks that need to be addressed immediately are:
1. Find a way to dynamically load HTML imports from the test specs.
2. Create HTML snippets that Polymer can recognize and upgrade before the tests run.
To this end, we’ve isolated some practices that have worked in the past into the Karma plugin karma-webcomponent-helpers. It’s a minimal set of assistant helpers that will probably grow as necessary.
Making Karma aware of what files to automatically load and serve is also accomplished a little differently than normal. Since we’re not loading script files as our source, we can’t use Karma’s default configuration. Our files config ends up looking like this:
// list of files/patterns to load in the browser
files: [
'components/platform/platform.js',
{
pattern:'components/src/**/*',
included: false,
watched: true,
served: true
},
{
pattern:'components/**/*.html',
included: false,
watched: false,
served: true
},
{
pattern:'components/**/*.css',
included: false,
watched: false,
served: true
},
{
pattern:'components/**/*.js',
included: false,
watched: false,
served: true
},
'test/**/*.spec.js'
],
The first and most obvious thing to note are the first and last lines: we’re loading platform.js first in order to ensure polyfills are loaded, and the last line includes all the test scripts that actually define our test JavaScript, with assertions and all.
The middle four lines define a wide net of stuff that Karma should be aware of but shouldn’t actually include in our template HTML (included: false). The first line is a pattern that matches everything in our components/src/ directory and lets Karma know it should watch those files for changes (watched: true). All of the patterns also specify served: true, meaning that, if we ask for a resource, Karma should serve it on its own HTTP server. This allows us to import HTML and have the browser ask for whatever is referenced without us explicitly defining it here.
For those familiar with unit testing browser code, this probably looked ugly until you got to that last line. This method of dependency management means that we can just import our HTML file, and it will include everything it needs to in order to run properly. Without this, we’d need to make sure we manually included and determined the order of third-party libraries like Jenga and jQuery along with our internal source. This can easily lead to lists dozens of lines long with sensitive ordering dependencies that need to be managed in parallel with the code and other test pages themselves.
Our entire karma.conf.js follows. The only other major changes to the default Karma configuration are to the plugins, browsers, and frameworks properties:
module.exports = function(config) {
config.set({
// base path that will be used to resolve all patterns (e.g. files, exclude)
basePath: '',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: ['mocha', 'chai', 'webcomponent-helpers'],
polymerTest: {
},
// list of files/patterns to load in the browser
files: [
'components/platform/platform.js',
'test/**/*.spec.js',
{
pattern:'components/src/**/*',
included: false,
watched: true,
served: true
},
{
pattern:'components/**/*.html',
included: false,
watched: false,
served: true
},
{
pattern:'components/**/*.css',
included: false,
watched: false,
served: true
},
{
pattern:'components/**/*.js',
included: false,
watched: false,
served: true
}
],
// list of files to exclude
exclude: [
],
// preprocess matching files before serving them to the browser
// available preprocessors:
// https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: ['progress'],
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values:
// config.LOG_DISABLE
// config.LOG_ERROR
// config.LOG_WARN
// config.LOG_INFO
// config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable/disable watching file and executing tests when any file changes
autoWatch: true,
// start these browsers
// available browser launchers:
// https://npmjs.org/browse/keyword/karma-launcher
browsers: ['Chrome', 'Firefox'],
plugins: [
'karma-webcomponent-helpers',
'karma-mocha',
'karma-chai',
'karma-chrome-launcher',
'karma-firefox-launcher'
],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests, and exits
singleRun: false
});
};
Test Specs
Our tests largely follow standard Mocha design with some boilerplate setup and teardown. If you’re unfamiliar with Mocha, it is designed to follow behavior-driven development (BDD) practices and has functions named as such. Mocha itself is a fairly lightweight harness for the organization of blocks that include assertions. Every spec starts off as a description of what feature is being tested:
describe('<x-dialog>', function () {
});
The karma-webcomponent-helpers plugin provides us with some useful functions to both import an HTML document (as a proper HTML import) and create elements. Nothing fancy, but both things we’ll want to do regularly when testing web components. At the start of our tests we’ll need to import our custom component. That can be done with Mocha’s before method, which runs once at the start of its describe block:
describe('<x-dialog>', function () {
before(function(done){
helpers.importHref('./base/components/src/x-dialog.html', function () {
Polymer.whenPolymerReady(done);
});
});
});
The helpers.injectElement(url, callback) function takes a callback that is called when the load event is dispatched on the related <link> element. Since we want to wait for our import to load before running our tests, we make this an asynchronous task simply by specifying a function argument as part of what is passed to before. Mocha will assume any of its methods that take in an argument are asynchronous and will wait for that argument to be called before moving on. This is important to note because it is not uncommon during refactoring to eliminate the need for some asynchronous call, thus deleting the subsequent call to done, which then hangs your tests without much description of why. We also ensure we are avoiding all the race conditions we can by delegating our asynchronous completion function to Polymer’s whenPolymerReadymethod. This is a handy method that is analogous to the globally dispatched polymer-ready event, with the added sugar of immediately calling the passed handler if polymer-ready has already been dispatched.
In order to create an element that we can test appropriately we can use another helper method, helpers.injectElement(snippet, callback), which will inject the passed HTML into the body of the document, return the created element, and run the callback after execution has been yielded to the browser for processing.
The creation of the element should be done before actual tests are executed and the element should be destroyed after every test in order to ensure the state is consistent and uncontaminated for each test scenario:
describe('<x-dialog>', function () {
var element;
before(function(done){
helpers.importHref(
'./base/components/src/x-dialog.html', function () {
Polymer.whenPolymerReady(done);
});
});
beforeEach(function(done){
element = helpers.injectElement(
'<x-dialog title="Hello World">Content</x-dialog>', done);
});
afterEach(function () {
element.remove();
})
});
Now the structure is well set up for a decent romp through unit testing. Tests are written as part of it blocks, which encourages the test author to describe the tests from the perspective of interaction, not the perspective of the developer. This sounds trivial, but it’s a valuable perspective shift that allows you to focus on what’s important, not what actually exists. Consider this example test plan:
1. Test the ignition.
2. Test the engine.
3. Test the tire capacity and durability.
versus this one:
1. It should turn on when you pull the pull cord.
2. It should spin the blade when the engine is on.
3. It should be able to withstand bumping into rocks and walls.
This is an obviously contrived example, but the second method of explaining what is being tested better outlines what is important in the tests so that it can be focused on. Getting back to our example, we can specify the following tests:
it('should instantiate via a constructor', function () {
var dialog = new XDialog();
expect(dialog).to.be.an.instanceof(HTMLElement)
});
it('should have a title', function () {
expect(element).to.be.an.instanceof(HTMLElement)
assert.equal(element.title, 'Hello World');
});
it('should open without throwing an error', function () {
element.show();
});
Notice how we’re testing the most minimal set of functionality per test. This ensures that failures are obvious in our scope and also reduces the effort necessary to write tests. Sure, these tests could be made better, but they serve as an example as to what can be done here. The full test spec looks like this:
describe('<x-dialog>', function () {
var element;
before(function(done){
helpers.importHref('./base/components/src/x-dialog.html', function () {
Polymer.whenPolymerReady(done);
});
});
beforeEach(function(done){
element = helpers.injectElement(
'<x-dialog title="Hello World">Content</x-dialog>', done);
});
it('should instantiate via a constructor', function () {
var dialog = new XDialog();
expect(dialog).to.be.an.instanceof(HTMLElement)
});
it('should have a title', function () {
expect(element).to.be.an.instanceof(HTMLElement)
assert.equal(element.title, 'Hello World');
});
it('should open', function () {
element.show();
});
afterEach(function () {
element.remove();
})
});
Running Our Tests
Our tests can be run via the command line simply by executing karma start (output truncated where necessary to fit within the page margins):
$ karma start
INFO [karma]: Karma v0.12.19 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [launcher]: Starting browser Firefox
INFO [Firefox 31.0.0 (Mac OS X 10.9)]: Connected on socket fsQW with id
INFO [Chrome 36.0.1985 (Mac OS X 10.9.2)]: Connected on socket Gl7a with id …
Firefox 31.0.0 (Mac OS X 10.9) WARN: 'platform.js is not the first script …
Chrome 36.0.1985 (Mac OS X 10.9.2): Executed 3 of 3 SUCCESS (0.201 secs … secs)
Firefox 31.0.0 (Mac OS X 10.9): Executed 3 of 3 SUCCESS (0.222 secs … secs)
TOTAL: 6 SUCCESS
This will also watch all of our source files, rerunning tests as anything changes. If we need to debug our code within a browser, we can click the “DEBUG” button in the upper righthand corner of the Karma test page that gets loaded in each of the spawned browsers (see Figure 16-1).
Figure 16-1. The Karma test page
This will bring you to a blank page allowing you to open the browser’s developer tools and inspect at will.
Running these tests manually is useful, but unit tests aren’t worth much unless they are run automatically. In the next chapter we’ll be incorporating our build process and will see how Karma and other testing tools can play a role as part of a larger task chain.
Summary
A lot of exciting promises have been made about web components. In this chapter, we faced one of the hard truths—lack of testability—and provided you with a possible route through. Solid unit tests are the foundation on which maintainable applications are made, and without them, teams can find themselves floundering around trying to keep their heads above water.
Unit tests automated via JavaScript and Karma are, in our eyes, the best way to test Polymerized web components right now. Tools will absolutely emerge to fill in the gaps, but this may be your largest pain point right now.
To be honest, though, there are many people and teams who don’t actually do very thorough unit testing. I know, it amazes even me, but it’s true. If you happen to be part of such a team (I won’t tell anybody), then I can assure you Polymer and web components will help the maintainability of your application. The concrete line of demarcation with web components—HTML tags, attributes, and associated public JavaScript methods—helps to encapsulate logic in a very strict manner. This strictness and the natural desire to adhere to it benefit everyone in the long run.