Writing Effective JavaScript Unit Tests with YUI Test

Writing Effective JavaScript Unit Tests with YUI Test

Functional testing, as opposed to unit testing, is designed to test the user’s experience with the product rather than input-output sets for code. If you find yourself wanting to test that the user interface responds in a specific way due to user interaction, then you really want to write some functional tests rather than unit tests. YUI Test can be used to write some basic functional tests, but the most popular (and quite good) tool for such testing is Selenium.

The best way to determine if something is a unit test is to ask if it can be written before the code that it’s designed to test actually exists. Unit tests, as part of test-driven development, are actually supposed to be written ahead of the actual code as a way to guide development efforts. Functional tests, on the other hand, cannot exist ahead of time because they are so tied to the user interface and how it changes in response to user interaction.

Each test suite can contain other test suites as well as test cases; only test cases can contain actual tests (methods beginning with the word "test"). The best way to organize your test hierarchy is to follow a very simple pattern:

* Create one test suite for every object you’re going to test.
* Create one test case for every method of an object you’re going to test and add it to the object’s test suite.
* Create one test in each test case for each input-output set.

The curious case of JavaScript unit testing

JavaScript’s dependent nature in the browser makes it difficult to accomplish true unit testing on anything but the lowest-level utility functions. JavaScript libraries are actually fairly easy to unit test because each method typically does one discrete operation given a certain set of inputs. The JavaScript library code doesn’t have any business logic or direct knowledge of the relationship between DOM elements, CSS, and the JavaScript itself. That’s why libraries such as YUI have such comprehensive unit test suites: the tests are pretty easy to write and then execute.

The larger problem is unit testing JavaScript code that runs web applications. This is where you start to run into serious dependency problems due to the interrelation HTML and CSS. The JavaScript code isn’t simply manipulating data; it’s expected to run within the web application environment. To do true unit testing, you would need to stub out the entire web application environment just to get the code to execute. And then, what do you test? A lot of the time you’re testing how the user interface responds to user input, which means you’re actually starting to cross over into the realm of functional testing (also called system testing).

Periodically, I get asked for help in running JavaScript unit tests on the command line using Rhino. While it is possible, I strongly recommend against doing this. If your JavaScript is intended to run in a web browser, then it should be tested in a web browser. Rhino is a completely different environment than any browser and, in fact, isn’t the JavaScript engine for any existing browser (it is a Java port of SpiderMonkey, the C-based library that was the JavaScript engine for Firefox prior to version 3.5). Testing JavaScript code in Rhino only tells you that the code works in Rhino, it does not tell you that the code runs in any browser.

Some folks have gone through a lot of trouble to try and bring command line JavaScript unit testing into the world. John Resig created env.js, a JavaScript utility that builds out a lot of the common browser environment in Rhino. As interesting as that is, you’re once again dealing with a browser environment that doesn’t exist in the wild. I have seen tests that work perfectly fine in all browsers and fail miserably in an env.js-powered Rhino environment. There’s no real value in testing code in an environment into which it won’t ultimately be deployed.

Even scarier is Crosscheck, a Java-based system that claims to test your code in several browsers without actually using the browser. Created by The Frontside Software, Inc., Crosscheck tries to recreate the browser environment of Internet Explorer 6, Firefox 1, and Firefox 1.5 in Java. As you might have expected, Crosscheck relies on Rhino as it’s JavaScript engine and then proceeds to build out each browser environment. An ambitious idea, for sure, but now you’re going one step further away from the truth: you’re relying on someone else’s understanding of browser quirks on which to base your tests. I’ve been in web development for a long time, but even I couldn’t sit down and list out every browser quirk. The result is that you’re testing in several mythical browser environments that have no real correlation to reality.

I’ll repeat, JavaScript code designed to be run in web browsers should be tested in web browsers. All code should be tested in the environment in which it is to be deployed. If your JavaScript code will be deployed to Rhino, then by all means, test in Rhino. But that’s the only reason you should test your JavaScript code in Rhino (or any other command line JavaScript engine).

It’s the automation, stupid

The real reason that command line tools keep trying to appear is for the purposes of automation. When the developer is sitting in front of his computer and running tests in browsers, the unit testing process is pretty simple. But that’s terribly redundant and, of course, boring. It would be much easier if the tests were automatically run periodically and the results were recorded. Really, the command line appeal is integrate test running into a continuous integration (CI) system.

The two CI systems I hear the most about are CruiseControl and Hudson. Both work in a similar manner, periodically running a series of tasks related to your build. They are capable of checking out code, running scripts, and of course, executing command-line operations. Command-line utilities fit perfectly into these systems because the output can easily be monitored for completion and errors. This represents a major problem since most of the browsers people use are GUI-based (Lynx is still around, though).

Fortunately, there is another movement of JavaScript testing focused on command line-initiated yet still browser-based testing. Leading the charge is Selenium, a tool primarily designed for functional testing is generally useful in that it can be run from the command line and can execute JavaScript inside of a browser. This means that, from the command line, you can use Selenium to fire up a browser, navigate to a particular page, run JavaScript commands, and inspect what happens to the page. What’s more, you can use Selenium Remote Control to fire up any number of browsers and perform the same tests. These results can be passed back into the command line interface, creating a seamless integration with CI systems. This is an area in which I’m currently doing more research. Stay tuned!

Another interesting tool that recently popped up is TestSwarm. TestSwarm’s approach is different than that of Selenium. Instead of manually starting browsers and navigating them to a page, TestSwarm relies on browsers to already be set up and attached to the TestSwarm server. The browsers can then poll the server to see if there are any new jobs that must be processed. The advantage is that you can add new browsers simply by opening a browser and pointing it to the TestSwarm server. Since the browsers are very loosely coupled to the system, upgrading to include new browsers is ridiculously simple.

TestSwarm also enables the crowd sourcing of tests. Anyone who wants to help test a product can joined a swarm and volunteer to leave the browser open for testing.