Automate acceptance tests with Selenium http://www-128.ibm.

com/developerworks/library/wa-selenium-ajax/

Automate acceptance tests with Selenium

How to use the Selenium test tool for functional testing of a Ruby on Rails and Ajax
application

Level: Intermediate

Christian Hellsten (christian.hellsten@fi.ibm.com), IT Specialist, IBM

20 Dec 2005

Acceptance, or functional, testing is designed to put manual tasks through their paces, but testing
these tasks by hand can be time consuming and prone to human error. In this article, the author shows
architects, developers, and testers how to use the Selenium testing tools to automate acceptance tests;
automating the tests saves times and helps eliminate tester mistakes. You also are provided with an
example of how to apply Selenium in a real-world project using Ruby on Rails and Ajax.

Acceptance testing of Web applications usually involves manual tasks like opening a browser and performing
actions described in a test case. Manually performed tasks are prone to operator error and are time consuming. It is,
therefore, a good practice to automate these tasks whenever possible to remove the human factor. This is where test
tools like Selenium come into the picture. Selenium helps you automate your acceptance tests and allows you to
build better tested and, hence, more reliable and maintainable software.

Acceptance testing, also known as black-box and functional testing, is a way of testing and verifying that an
application works according to functional, non-functional, and other important stakeholder requirements.
Acceptance tests complement unit and integration tests, which are usually written using xUnit frameworks.
Acceptance tests can also be written using a programming language, but Selenium and similar tools like Fitnesse
also support tests written using a tool-specific document format.

Acceptance tests differ from unit and integration tests in the following ways:
The application is tested as a whole, end-to-end entity, not just one class or a set of classes as it is with unit
and integration tests.
Acceptance tests are performed against the user interface using, for example, a browser against a Web
application interface.
The person writing the test cases does not necessarily know about the internals of the application, hence the
designation of black-box testing. Non-technical users can write acceptance tests.

A bit of background
Before going further into Selenium, I'd like to offer some background on the following three topics, since they are
relevant to this article even though they're not the central theme:
Continuous integration
Ajax
Ruby/Ruby on Rails
Continuous integration
Continuous integration aims at automating the build and test processes so that they run automatically once or many
times a day instead of, say, manually once a month. The biggest benefit of using continuous integration is that code
changes are integrated automatically on a regular basis. And if the system is broken and does not build
successfully, continuous integration tools like Apache Continuum and Luntbuild can automatically notify the team
by sending an e-mail (see Resources).

Ajax
Ajax stands for Asynchronous JavaScript and XML; it is a newly coined term for relatively old technology. The
main idea behind Ajax is that the Web application responds more quickly to user actions because only parts of the
page need to be updated instead of the whole page.

1 of 10 2007-8-30 3:59
Automate acceptance tests with Selenium http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

Ajax introduces more complexity to Web applications that is also reflected in testing. This is because Ajax, as the
name implies, uses JavaScript and asynchronous HTTP requests to update the page contents. Each browser
contains small differences in implementation compared to others. Selenium makes a perfect tool for testing and
detecting these differences since it runs inside the most popular browsers.

Ruby/Ruby on Rails
Ruby is an open-source interpreted scripting language for quick and easy object-oriented programming; it features
a wide range of libraries and is simple, straight-forward, extensible, and portable. The language was created by
Yukihiro "Matz" Matsumoto and was designed to make programmers happy by allowing them to concentrate more
on the task at hand than syntax.

Rails is a full-stack, open source Ruby Web framework created by David Heinemeier Hansson. Rails is designed to
make writing real world applications less code-intensive and easier than J2EE and XML, for example. All the
layers are built to work seamlessly together so you can use a single language to write everything from templates to
control flow to business logic. Rails uses YAML instead of XML configuration files and reflection and runtime
extensions in favor of annotations. And there is no compilation phase -- you make a change and watch it work.

What is Selenium?
Selenium is an acceptance testing tool written by ThoughtWorks specifically for Web applications. The Selenium
home page says the following is the biggest benefit of using Selenium when compared with other test tools:

Selenium tests run directly in a browser, just as real users do. And they run in Internet Explorer,
Mozilla, and Firefox on Windows, Linux, and Macintosh. No other test tool covers such a wide array
of platforms.

Note that IBM's Rational Functional Tester tools perform these tests as well, but at a much more complex and
wide-reaching level, and come with IBM support. Selenium, on the other hand, prioritizes one task -- a Web
application -- and can perform with more agility.

Additional benefits of using Selenium and running tests inside a browser are many. Here are two main ones:
You test the application from the end user's perspective by writing Selenium test scripts that mimic user
actions.
You can identify browser incompatibilities more easily by running the tests in different browsers.
The core of Selenium, also referred to as the browser bot, is written in JavaScript. This allows test scripts to run
inside supported browsers. The browser bot is responsible for executing commands received from test scripts
which are written in either HTML using a table layout or a supported programming language.

Selenium works with the following:

Internet Explorer Mozilla Firefox Safari

Windows XP 6.0 1.6+, 1.7+ 0.8+, 0.9+, 1.0
Red Hat Linux 1.6+, 1.7+ 0.8+, 0.9+, 1.0+
Mac OS X 10.3 not supported 1.6+, 1.7+ 0.8+, 0.9+, 1.0+ 1.3+

Selenium commands
With Selenium commands,
the script writer describes
the actions that the browser
bot is to perform in the
Selenium and the Rational Functional Tester
Companies or organizations with bigger projects typically need a standard product
with many features and reliable support. All projects within the organisation then

2 of 10 2007-8-30 3:59
Automate acceptance tests with Selenium http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

browser. You can split use this product. One solution is the Rational Functional Tester, rather than
these commands into two Selenium, which is purposely not standardized and is limited in its design to
groups -- actions and address much smaller tasks with ease.
assertions:
Rational Functional Tester is used with such applications as .NET and Java GUIs,
Actions simulate the
Siebel, Terminal-based applications, and, of course, Web applications. The Tester
user's interaction
contains thousands of features not available in Selenium, including Test Manager
with the Web
and ScriptAssure. It's integrated with version control and has tools and components
application. Clicking
that extend its use to many areas, and is used by all projects within the same
a button and filling
organization.
out a form are both
examples of Get more specs on this advanced test automation at
common user http://www3.software.ibm.com/ibmdl/pub/software/rational/web/datasheets/rft.pdf.
actions that you can
automate with
Selenium
commands.
Assertions verify the expected outcome of a command. Common assertions include verifying that the page
content or that the current location is correct.
You can find a complete list of available commands on the Selenium Web site (see Resources).

Selenium modes
You can use Selenium in two modes: test runner and driven. The two modes differ in complexity and the way they
are written. Driven test scripts tend to be more complex to write since they are written in a programming language.
The difference in complexity is minimal if you use a high-level dynamic programming language like Python or
Ruby.

The biggest difference between the two modes is that the tests are partly run from outside the browser if using
driven scripts in which test runner scripts are run completely inside the browser.

Both test runner and driven test cases can be integrated with continuous integration tools.

Test runner mode

Selenium test runner scripts, also referred to as test cases, are written in HTML using a simple table layout shown
in Listing 1.

Listing 1. Selenium test case structure

<table border="1">
<tr>
<td>First command</td>
<td>Target</td>
<td>Value</td>
</tr>
<tr>
<td>Second command</td>
<td>Target</td>
<td>Value</td>
</tr>
</table>

Test runner scripts are usually deployed on the same server as the application under test (AUT). This is because the
browser bot uses JavaScripts to emulate user actions. These scripts run in a restricted sandbox environment. If you

3 of 10 2007-8-30 3:59
Automate acceptance tests with Selenium http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

need to circumvent these restrictions, use a proxy.

Test runner scripts use the same concepts of test suite and test case as xUnit frameworks. Both test cases and
commands execute sequentially in the order they appear in test suites and test cases. In Listing 1:
The first column contains the command or assertion.
The second column contains the target for the command or assertion. Specify the target with one of the
many supported element locators. Usually the ID or name of the element is used, but also XPath and DOM
locators are supported.
The third column contains the value which specifies parameters for commands or assertions. This can be the
desired value for a text field when using the type command.
Test runner scripts are easy to read and write even for non-technical people. The example in Listing 1 will, when
opened in a browser, look like this table:

First command Target Value

Second command Target Value
Next I'll describe how to write a simple but complete test case using both commands and assertions.

Example test case

The test script in Listing 2 performs the following actions upon execution:
1. It opens the change address page by going to /change_address_form.html.
2. It types Betelgeuse state prison in the text box that has the ID address_field.
3. It clicks on the input field named Submit. Note this uses XPath to find the Submit button, which sends the
form data to the server.
4. It verifies that the page contains the text Address change successful.

Listing 2. Example of using commands and assertions in a test case

<table>
<tr>
<td>open</td>
<td>/change_address_form.html</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>address_field</td>
<td>Betelgeuse state prison</td>
</tr>
<tr>
<td>clickAndWait</td>
<td>//input[@name='Submit']</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td>Address change successful</td>
<td></td>
</tr>
</table>

Test suites
Acquiring full test coverage of an application usually requires more than one test case. This is why Selenium uses
the concept of test suites. Test suites are used to group a set of test cases with similar functionality together so that
they can be run in sequential order.

Test suites are written in the same way as the test cases, using simple HTML tables. The default test suite executed
by Selenium is named TestSuite.html. Listing 3 shows a test suite that tests an application in the same way a
typical user would. Note that test suites use a single-column table and that each row points to a file containing a
test case.

4 of 10 2007-8-30 3:59
Automate acceptance tests with Selenium http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

Listing 3. Test suite example

<table>
<tr>
<td>Test suite for the whole application</td>
</tr>
<tr>
<td><a href="test_main_page.html">Access main page</a></td>
</tr>
<tr>
<td><a href="test_login.html">Login to application</a></td>
</tr>
<tr>
<td><a href="test_address_change.html">Change address</a></td>
</tr>
<tr>
<td><a href="test_logout.html">Logout from application</a></td>
</tr>
</table>

Next I'll look at driven test scripts.

Driven mode
Driven Selenium scripts are written in one of the supported programming languages -- currently Java, Ruby, and
Python drivers are available. The scripts run in a separate process outside of the browser. The driver's task is to
execute the test script and drive the browser by communicating with the browser bot which runs inside the
browser. The communication between the driver and the browser bot uses a simple Selenium-specific wire
language called Selenese.

Driven scripts are more powerful and flexible than test runner scripts and you can integrate them with xUnit
frameworks. The downside to driven scripts (when compared with test runner scripts) is that they are more
complex to deploy and write. This is because the driver has to perform the following tasks:
Start the server.
Deploy application under test (AUT).
Deploy test scripts.
Start the browser.
Send commands to the browser bot.
Verify the result of commands executed by the browser bot.
Driven scripts are also more dependant on the application runtime environment. For example, the Java driver uses
an embedded Jetty or Tomcat instance to deploy the application under test. Currently, an effort is in progress to
integrate Selenium into Ruby on Rails, but at the time of writing it was not yet released.

Listing 4 is an excerpt from a driven test script that uses the Ruby driver. Note that I left out the steps for
instantiating the server and browser and that the test script code is almost as simple as a test runner script.

Listing 4. Example of using the Ruby driver

.
.
puts selenium.open('/logout.html')
puts selenium.verify_location('/index.html')
.
.

Real-life requirements

5 of 10 2007-8-30 3:59
Automate acceptance tests with Selenium http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

In the next two major sections (Real-life requirements and Real-world use cases), I describe how to use Selenium
in a real-world scenario and write Selenium test cases for a simple stock quotes viewer application using Ruby on
Rails and a bit of Ajax technology. Although the application is written using Ruby on Rails, you can apply the
example to any Web application since the test scripts are written in HTML for the test runner mode. The sample
application will probably work with older or later versions, but it was tested with Ruby 1.8.3 and Ruby on Rails
0.14.2.

Ruby is usually included in the distribution if you have Linux. Run ruby -v from a command prompt to check
what version you have. You can find a Ruby distribution for most platforms at http://www.ruby-lang.org/.

Your next step is to install Ruby on Rails through the RubyGems packaging system. To do this, simply execute
gem install rails --include-dependencies. On some platforms you have to perform some extra
steps, so check the Ruby on Rails Web site for more details.

The current Selenium version available, at the time of writing, is 0.6. I've already integrated it into the sample
application (see Download) by downloading the Selenium Core package from http://selenium.thoughtworks.com/
and copying the folder named selenium to the folder that serves static content. In Ruby on Rails applications, the
name of this folder public. In a J2EE Web application, you put it in the root of the Web application or WAR
archive file.

Your last step is to download the example application; access the package from Download. Extract the application
and open a command prompt. Then change directory to where you extracted the application. To start the
application, run ruby script/server. You should see that Rails started without errors as shown in Figure 1.

Figure 1. Running Ruby on Rails from command prompt

Real-world use cases

In this section I'll list the use cases for the sample application. With these simplified use cases, you can write
acceptance tests that simulate the steps performed by the user and verify that the result of these is as expected. The
stock quotes application implements the following four use cases:
Login
View stocks
View stock details
Logout
The code implementing these use cases is already written; you can find it in the app directory and the test cases in
the public/selenium/tests folder.

Login use case

Most people know how a login page works -- you enter your username and password and submit the data to the
server. If the credentials are valid you are logged in and taken to the secured resource. In the sample application,
this test case involves the following user actions and assertions which you have to translate into a Selenium test
case:
1. Click login link.
2. Verify that the system requires that the user logs in.
3. Enter username.

6 of 10 2007-8-30 3:59
Automate acceptance tests with Selenium http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

4. Enter password.
5. Press login button.
6. Verify that login was successful.
Figure 2 shows the Selenium test case for these requirements. Note that I took the screenshot after running the test.
The green rows indicate assertions that were successfully verified.

Figure 2. Login and view stocks test case

View stocks use case

The view stocks page displays a list of companies. The test case for this is so simple that it was included at the end
of the previous test case. It verifies that the current location is /main/list_stocks and that the page contains the text
Click on a company name to view details.

View stock details use case

The view stock details use case is triggered on the view stocks page. A user's clicks on a company name triggers an
Ajax request to the server. The response from the server contains the company details and will be inserted into the
current page without a complete page reload. The test case for this use case performs the following user actions:
1. Click on company name Acme Oil.
2. Verify that the company details are displayed on the page.
3. Click on company name Acme Automotive.
4. Verify that the company details are displayed on the page.
The request happens asynchronously because Ajax is used. This poses a different challenge than with normal Web
applications where everything typically happens synchronously. You can test Ajax functionality in the same way
as other functionality. The only exception is that you have to tell Selenium to pause and wait for the Ajax
command to finish. For this, you will use the pause command to wait for the Ajax command to finish. You can
also use the waitForValue and waitForCondition commands instead of pause as Joseph Moore points out
in his recent blog post (see Resources).

Figure 3 shows the requirements translated into a Selenium use case.

Figure 3. View stock details test case

Note the pause commands: These are required to allow the asynchronous request to finish and update the page
content. Without the 500 millisecond pause, the test fails (shown in Figure 4).

Figure 4. Failed view stock details test case

7 of 10 2007-8-30 3:59
Automate acceptance tests with Selenium http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

The pause command also tests the non-functional requirements for the Ajax functionality. 500 milliseconds is a
good value for the pause command since Ajax requests should execute and finish quickly. You can try to remove
the pause commands and see what happens. If the tests fail on your machine, try to increase the value to 1000
milliseconds.

Logout use case

The logout use case is easy to implement and is summarized by the following two steps:
1. Click logout link.
2. Verify that logout was successful.
Figure 5 shows the final test case.

Figure 5. Logout test case

All tests are added to the default test suite which is shown to the left of Figure 6.

Figure 6. Test suite for sample application

Executing the test suite

The last thing you do is execute the whole test suite in both Mozilla Firefox and Microsoft Internet Explorer. To do
this, open http://localhost:3000/selenium/TestRunner.html in the browser and click on the All
button shown in Figure 6. Tests cases and assertions that fail are colored red, but in your case they should all
complete successfully in both browsers (also shown in Figure 6). Note that I used Mozilla Firefox 1.0.7 and
Internet Explorer 6.0.

You can also walk or step through the test suite which means Selenium will execute the test suite slowly enough
for you to see each step of the test suite as it executes in the browser.

Conclusion
Selenium is a useful and important addition to the toolbox of software engineers, designers, and testers. Together

8 of 10 2007-8-30 3:59
Automate acceptance tests with Selenium http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

with a continuous integration tool, it allows teams to automate acceptance tests and build better software as they
find bugs easier, earlier, and more often. An added benefit of Selenium is that it saves time and allows the team to
concentrate on more valuable activities by freeing developers and testers from manual tasks which can and should
be automated.

Download
Description Name Size Download method
Sample application wa-selenium-ajax-example.zip 286KB HTTP

Information about download methods

Resources
Learn
Review a complete list of available Selenium commands.

See Wikipedia's definition of acceptance tests.

Discover how a Selenium integration into Ruby on Rails looks in this brainstorming session.

See Wikipedia's definition of Ajax.

Try Selenium's waitForValue and waitForCondition commands -- Joseph Moore's alternative to

the pause command.

Dig into these Ajax resources on developerworks:

Build apps with Asynchronous JavaScript with XML, or Ajax (November 2005) demonstrates how to
construct real-time-validation-enabled Web applications with Ajax.
Ajax for Java developers: Build dynamic Java applications (September 2005) is a guide to
application development using Ajax.
Ajax and scripting Web services with E4X (April 2005) is a primer for Ajax.
Ajax for Java developers: Ajax with Direct Web Remoting (November 2005) demonstrates how to
automate the heavy-lifting of Ajax.

Review the archived Diagnosing Java code series (developerWorks, 2001) includes automating testing tasks
as a matter of best programming and debugging practices.

Explore all Ruby and Ruby on Rails articles at developerWorks, including these gems:
The tutorial Sockets programming in Ruby teaches Ruby basics plus the most important classes to
develop sockets-based networking applications using the Ruby language (October 2005).
Fast-track your Web apps with Ruby on Rails (June 2005) describes a framework for rapid
development using an MVC pattern.
Ruby on Rails and J2EE: Is there room for both? (July 2005) compares Ruby on Rails and J2EE.

Check out all the Python articles at developerWorks.

Get the white paper on Advancing Toward Test Automation through Effective Manual Testing as well as
the specifications for the Rational Functional Tester.

Get products and technologies

Get details on the Selenium tool, built by ThoughtWorks in accordance to Agile development methods for

9 of 10 2007-8-30 3:59
Automate acceptance tests with Selenium http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

enterprise software development.

Visit the Ruby on Rails homepage and the Ruby language homepage for everything you need to get up to
speed with these technologies.

Check out Continuum, a continuous integration server for building Java-based projects.

Explore Luntbuild, a powerful build automation and management tool designed to automate builds and act
as a build management system.

Try Fitnesse for an acceptance testing framework.

Get the Mozilla Firefox browser.

Get Selenium Recorder, a Firefox extension that allows you to record Selenium test scripts.

Try out the Rational Functional Tester with this free 30-day trial download.

Discuss
Participate in the discussion forum.

About the author

Christian Hellsten is currently an IT Specialist with IBM Business Consulting Services. He has more than six
years of experience in the IT industry working on large-scale e-business projects. His area of expertise is in Java,
J2EE and EAI architecture, design and development. He also enjoys working with more dynamic, productive, and
fun languages and frameworks like Python, Ruby, and Ruby on Rails.

10 of 10 2007-8-30 3:59