Functional Testing in PHP using Selenium IDE

What a title, eh? Yeah, it might sound a bit like buzzword after buzzword, but functional testing in PHP is a serious topic - sometimes it means the difference between spending fifteen minutes doing a bug check and several hours depending on the size of your site. Now, it’s important to mention that functional tests are different from unit tests - we’re not going to be testing individual functions or methods, but rather we’re going to be making sure that the site works the way our users expect it to from their point of view. This means that we can check our site for error messages and verify that they pop up when they should, and stay hidden when they shouldn’t.

What’s Functional Testing?

Functional testing is fairly easy to define when you have an idea of what unit testing is, because they’re different edges of the same sword (both of which can cut you given that you don’t use them properly). Unit testing is the practice of sitting down with your code and asking, “What can I put into this function/method and what should it give me back?” You then write a bunch of tests to make sure that the function or method returns the proper values with good data that you give it, and fails when you give it bad data. Functional testing is similar in that we’re checking a bunch of “functions” for what we put in versus what we get out, though this kind of testing is a bit more “big picture” than that - we’re testing how the user is going to be interacting with the program. For this line of testing, checking for improper input is usually more important than checking that the “good” input is working, which might not exactly be the case in unit testing.
In PHP, functional testing equates to a few steps.

1. Identify “stories” of what the user can do - “The user can change their password” or maybe “The user can perform a keyword search.” It doesn’t have to be a super-detailed or difficult task.
2. Break down that “story” into a series of actions. Sometimes this means sub-stories, and sometimes it’s as simple as “The user clicks on the link to the search page. The search page loads. The user types in their query. The user clicks the ’search’ button. The user finds their results.”. Sub-stories would define another test to write, since you want to test your web application for small enough details to make the tests useful. If you test your entire site all at once, it’s likely going to be difficult to identify what went wrong.
3. Write the tests - we’ll be showing you how to do this in just a little bit by using the Selenium IDE Firefox extension, but there are quite a few tools that could also help you perform these tasks.
4. Run the tests. In fact, do it a couple times. Run the test every time you make a change that might have affected the section you wrote the test for. This is where the real value of functional testing comes in - you don’t have to go check that your entire site still works by hand when you changed that database connecting script. Anything in your site that your code change may have affected should be covered by the tests you’ve written, so you know exactly what’s not working anymore.

The Tools
For PHP functional testing, there’s nothing better to me than OpenQA’s Selenium platform. This entails a suite of different tools with tests that can be written in a multitude of different languages. The two I use most frequently are Selenium IDE (the Firefox extension), and Selenium RC. Go ahead and download them both - though make sure to bookmark this page before you reset Firefox so you can come back to my wonderful world of functional testing.
Building a test in Selenium IDE
For this brief introduction to writing tests, we’re going to do a simple Google query. Our test story would be something like, “The user can perform a search and view appropriate results.” Easy, right? So let’s get started.

1. Install the Selenium IDE Firefox extension from the link above.
2. Go to “Tools” -> “Selenium IDE”. You should see this screen:
   
3. Stop recording by pressing the big red circle, and open up a new tab. Go to http://www.google.com and start recording again.
4. With Selenium IDE in “Record” mode, click on the search box and type in “hello world” or whatever overused internet meme that comes to mind, then click the search button.
5. Upon finding the results, highlight any phrase you find on the page that you think should be there (if you searched for “all your base”, highlight “are belong to us”). Right click your selection and you should see that Selenium has added some new options for you, one of which being “verifyTextPresent”. Go ahead and click on it.
   
6. Stop recording. This will be our entire test. I bet you’re sitting there thinking, “Wait, that’s it? That’s stupid!” It’s not stupid! It’s really that easy. The point is that we’ve just proven that the search function works. Go ahead and hit the “Play” button to prove it to yourself. If everything comes up green, then the test passed and you can move onto the next test (so long as you didn’t use ‘assertTextPresent’ instead). The entire purpose of these tests is to string them together in what we call a “Test Suite” that can test all of the functionality of your site without you having to lift a finger. Save this test somewhere as myTest.html or some other cliche filename.
   There’s a distinct difference between ‘assertTextPresent’ and ‘verifyTextPresent’. This is because an assert will stop the test if that particular check fails, while a verify command will note the failure and attempt to continue. There are different purposes for both, though I find that I personally use verify commands much more often than asserts.

Using Selenium RC to automate a Selenium Test Suite
So we have a little test now. Maybe you’ve created a couple because you’re insane and making test is that much fun! Uh huh.. Anyway, you’re going to need a way to put all of these tests together into a package that runs on its own. We call this package a “Test Suite”, and Selenium IDE won’t cover doing more than one test at a time. We need a new tool for this, so I hope you’ve downloaded Selenium RC from the link above.

First thing’s first - you need to make sure you’ve got Java Runtime Environment 1.5 or above installed on your system. You can check this by opening your command line console (Start, Run, ‘cmd’) and typing the command “java -version” and hitting enter. If it comes back with something along the lines of “java version 1.6.2″ then you’re all set. If it doesn’t understand what you’re asking it, or you have an older version of the JRE, you can download the newest version here.

1. Before we can run tests, we need actual test suite code. It’s not obvious from the Selenium documentation what this looks like, so I’ve adapted a test suite from the Selenium Core package to use as an example that you can download here (Right click, save as - rename it to .html instead of .txt).
2. Copy that test suite and your myTest.html file to wherever you extracted your Selenium RC package to. Within the Selenium RC package should be several folders, one of which contains the term “selenium-server”. This is where you ought to put your files for now - just for this test. Later on, you’ll probably want to become a bit more organized with it.
3. Open up your command line and cd to your Selenium server folder you just put those files in.
   Mine’s “C:\Selenium\selenium-server-0.9.2″.
4. Run the following command on your command line:
   java -jar selenium-server.jar -htmlSuite
   “*firefox” “http://www.google.com”
   “c:\Selenium\selenium-server-0.9.2\TestSuite.html”
   “c:\Selenium\selenium-server-0.9.2\results.html”
   What’s going on here is that we’re telling java to run selenium with an htmlSuite. That htmlSuite option takes parameters for browser, base url (the place to start the tests at), the absolute path to your test suite, and an absolute path to save results to.
5. The test suite should complete, and you ought to have a new file in your Selenium server folder called “results.html”. Open up this file and it should look a bit like this:
   

This test suite only contained one test, but adding them are as easy as duplicating the table row in the sample test suite above and changing the path to whatever new test you want to run.
Applying this to practice
As a PHP developer, I find that some projects I’m assigned to require that I work on many fragile houses of cards wherein changing one little piece of code can break something on the other side of the site. Functional testing for these PHP scripts becomes not only useful, but rather important and required. Debugging your site with a functional test suite is about eleventy billion times more time efficient than doing it by hand several times a day. Build yourself a detailed test suite and check every form or action you do for both pass and failure cases - and run them after any significant changes, preferably daily. It will seem like a waste of time at first, but I guarantee you that the quality of the code you write will be better and you won’t be stuck with half-broken sites nearly as often.
How about you? How do you see functional testing affecting you, and if you’ve put it in practice, how’s it helped or hurt you?