Writing selenium tests in Java for XWiki Products

Writing selenium tests in Java for XWiki Products

This document is a short tutorial to get started writing integration tests for XWiki products, using the Java Selenium test framework. Selenium tests are integration tests that reproduces human interaction with a web application executing mouse and keyboard commands in a web browser.

The first part of the tutorial explains how to record tests scenarios using the Selenium IDE firefox extension. The second part deals on how to translate these scenarios to actual Java tests cases that can be integrated in a distribution-test module of a XWiki product.

The tutorial take a test on the rollback feature of XWiki as an illustration. To get more examples, tests can be found under the /distribution-test/selenium-tests/ directory of each XWiki product, for example here for XWiki Enterprise.

Recording a test with the Selenium IDE

Installing and running the Selenium IDE

The Selenium IDE comes as a firefox extension. It allows to "to easily and quickly record and play back tests in the actual environment that they will run". ^[1] You can download it from here. Once installed, it can be found under the "Tools" tab of Firefox.

Recording a test case

With the Selenium IDE installed, it is possible record and replay test scenarios. Ours will consist of opening the home page of a XE installation, authenticate as Admin, edit the Main.WebHome page with some dummy content (, and save, and then rollback it's version to the original 1.1. To verify the text passes, we will finally select "Welcome to your wiki" from the original Main.WebHome we have roll-backed right-click and choose "verifyTextPresent Welcome to your wiki" in the list.

The selenium IDE records the following commands :

This scenario recorded, it is possible to save it, and make the selenium IDE replay it in firefox.

Screenshot of the Selenium IDE replaying the scenario in Firefox

Writing the test in Java for XWiki

Now with this record, we have a good idea of what our final test should consist of. The next step is to translate it as java code, so that it becomes part of the build process and can be ran on continuous integration servers. XWiki uses the maven selenium plugin to start and stop the Selenium Remote Control server.

XWiki wraps the selenium API for its own needs, and offers a test framework on top of it to facilitate XWiki product testing. For our rollback test, we can extend the AbstractXWikiTestCase from the com.xpn.xwiki.it.selenium.framework package of XWiki Enterprise distribution-test module. We will ensure our test runs authenticated by calling the loginAsAdmin() method in the setUp() of the test. Then we can write a testRollbackToFirstVersion() that reproduce the actions we have recorded with the Selenium IDE :

public void testRollbackToFirstVersion() throws Exception
    {
        open("Main","WebHome","edit","editor=wiki");
        setFieldValue("content", "aaa");
        clickEditSaveAndView();
        open("Main","WebHome","rollback","rev=1.1");
        clickLinkWithLocator("//input[@value='yes']");
        assertTextPresent("Welcome to your wiki");
    }

We first open the Main.WebHome page in edit mode, with the wiki editor. We enter some dummy content in the content field of the form, then click the "save and view" button. As we can see, the XWiki Selenium wrapper offers facilities methods to do such actions easily. For the actual rollback, we don't need to go through the menu clicking, as we can directly open the URL that handle this action, and then just click yes on the confirmation screen, using a Locator to tell Selenium which HTML element to click on (here, we use a XPath Locator, but other strategies are possible. You can read more about Selenium Locators here). Finally, we have to ensure our test passes, by checking the original text is back on the document, using the assertTextPresent method.

Our complete test class will look like the following :

package com.xpn.xwiki.it.selenium;

import com.xpn.xwiki.it.selenium.framework.AbstractXWikiTestCase;
import com.xpn.xwiki.it.selenium.framework.AlbatrossSkinExecutor;
import com.xpn.xwiki.it.selenium.framework.XWikiTestSuite;

import junit.framework.Test;

/**
 * Verify versioning features of documents and attachments.
 *
 * @version $Id: $
 */
public class VersionTest extends AbstractXWikiTestCase
{
    public static Test suite()
    {
        XWikiTestSuite suite =
            new XWikiTestSuite("Verify versioning features of documents and attachments");
        suite.addTestSuite(VersionTest.class, AlbatrossSkinExecutor.class);
        return suite;
    }

    protected void setUp() throws Exception
    {
        super.setUp();
        loginAsAdmin();
    }

    /**
     * Verify we can rollback to the first version of a document that is bundled in the default
     * distribution.
     */ 
    public void testRollbackToFirstVersion() throws Exception
    {
        open("Main","WebHome","edit","editor=wiki");
        setFieldValue("content", "aaa");
        clickEditSaveAndView();
        open("Main","WebHome","rollback","rev=1.1");
        clickLinkWithLocator("//input[@value='yes']");
        assertTextPresent("Welcome to your wiki");
    }
}

We can the run our test typing "mvn install -Dpattern=VersionTest" under distribution-test/selenium-tests/ in XWiki Enterprise sources. Running the test will actually launch the Selenium Remote Control, that itself will execute the test live in a browser you will be able to see.

We still need to add it to the test suite of XWiki Enterprise, so that it is executed along with the other existing tests when performing a distribution test. We do so editing the AllTests class :

public static Test suite() throws Exception
    {
        TestSuite suite = new TestSuite();

        addTestCase(suite, DeletePageTest.class);
        [...]
        addTestCase(suite, VersionTest.class);
        [...]
        addTestCase(suite, WatchListTest.class);

        return new XWikiSeleniumTestSetup(new XWikiTestSetup(suite));
    }

Writing selenium tests in Java for XWiki that tests Ajax specific functionality

One of the best solution for testing the Ajax specific functionality is function waitForCondition(wrote by Dan Fabulich), that comes as extension. What for condition does is 'Waits for any arbitrary condition, by running a JavaScript snippet of your choosing. When the snippet evaluates to "true", we stop waiting'(Dan Fabulich words).

A quick example is this:

getSelenium().waitForCondition("selenium.page().bodyText().indexOf('Welcome to XWiki Watch') != -1;", "2000");

WaitForCondition() gives you access to a object called selenium(you can find the api in file selenium-api.js or here). Selenium object also gives access to an other object called browserbot(selenium.browserbot - api can be found in file selenium-browserbot.js or here). With the function page() the html page is return(including the head section) and with bodyText() we receive only the body section of html page. The function indexOf() search for the text into body text. While this condition is false waitForCondition() will re-execute the Javascript snippet until the condition is true or the 2 seconds are gone.

An other example:
In XWiki enterprise there is a section where you can see all the files(of XWiki application) in a table. There are some filters after page, space and last author. If you type one letter into page filter, files whom contain that letter will appear into the table. While testing into selenium this feature isn't enough to just type the letters into the form, you also have to refresh the element that loads the table according to the filter. In our case in ajax-loader.

private void fillTableFilter(String field, String text)
    {
        getSelenium().typeKeys(field, text);
        getSelenium().waitForCondition("selenium.browserbot.getCurrentWindow().document." +
            "getElementById(\"ajax-loader\").style.display == \"none\"", "3000");    
    }

Avoid Dependencies in Selenium Tests

A class of selenium tests contains more tests. For each of this tests the selenium remote control will launch a new instance of the browser so it's very important for the writer of the tests to avoid dependencies in this tests.

For example: If I test Scheduler feature from XWiki enterprise. In one test method I could create a Job for scheduler. And in the next one I can assume that the Job exist in Scheduler's Job List and start testing the edit/delete functionality of the Job. There is a high probability that the test will fail, because the second test depend on the first one. But what if the order of execution for this test is changed and the second one gets to be executed before the first one ? Then the test will fail.

public void testCreate()
{
	open("Scheduler","WebHome");
	setFieldValue("title", "xyz");
	clickLinkWithXPath("//input[@value='Add']");
	setFieldValue("XWiki.SchedulerJobClass_0_jobName", "Tester problem");
	setFieldValue("XWiki.SchedulerJobClass_0_jobDescription", "Tester problem");
	setFieldValue("XWiki.SchedulerJobClass_0_cron", "0 10 15 2008");
	getSelenium().click("formactionsave");
	this.waitPage(10000);
	clickLinkWithText("Back to the job list");
	assertElementPresent("//td[text()='Tester problem']");
}
	
public void testEditJob()
{
	open("Scheduler","xyz","inline");
	setFieldValue("XWiki.SchedulerJobClass_0_jobName", "Tester problem2");
	setFieldValue("XWiki.SchedulerJobClass_0_jobDescription", "Tester problem2");
	setFieldValue("XWiki.SchedulerJobClass_0_cron", "0 10 15 2009");
	getSelenium().click("formactionsave");
	waitPage(10000);
	clickLinkWithText("Back to the job list");
	assertElementPresent("//td[text()='Tester problem2']");
}

Solution: If you are testing something like Panel, Scheduler feature is good that in the setUp() method of the class(that contains the tests) to create it and in methods just to check if the Panel/Job is there.

public void setUp() throws Exception
{
   super.setUp();
   loginAsAdmin();
   open("Scheduler","WebHome");
   assertElementPresent("//h3[text()='Comments']");
   assertElementPresent("//h3[text()='Attachments']");
   //if the Job doesn't exist will be created
   if(!getSelenium().isTextPresent("Tester problem"))
   {
        setFieldValue("title", "xyz");
	clickLinkWithXPath("//input[@value='Add']");
	waitPage(10000);
	setFieldValue("XWiki.SchedulerJobClass_0_jobName", "Tester problem");
	setFieldValue("XWiki.SchedulerJobClass_0_jobDescription", "Tester problem");
	setFieldValue("XWiki.SchedulerJobClass_0_cron", "0 10 15 2008");
	getSelenium().click("formactionsave");
	waitPage(10000);
	clickLinkWithText("Back to the job list");
	assertElementPresent("//td[text()='Tester problem']");
   }
}

public void testEditJob()
{
    open("Scheduler","WebHome");
    getSelenium().click("//a[@href='/xwiki/bin/inline/Scheduler/xyz']");
    waitPage(10000);
    setFieldValue("XWiki.SchedulerJobClass_0_jobDescription", "Tester problem2");
    setFieldValue("XWiki.SchedulerJobClass_0_cron", "0 10 15 2009");
    getSelenium().click("formactionsave");
    waitPage(10000);
    clickLinkWithText("Back to the job list");
    assertElementPresent("//td[text()='Tester problem']");
}

Browser Launchers and their importance in Selenium Tests

When running selenium tests a Selenium Remote Control will open into a new browser window and an other browser window will be opened in which the tests will be ran. To open the browser window, Selenium needs browser launchers.

Selenium offers a few browser launchers, the target being: a browser launcher per operating system specific browser(Mozilla Firefox for Unix/Linux, Internet Explorer for Windows, Safari for MacOS and a few others like Konquer)^[2]. For those browsers that don't have browser launcher implemented, Selenium offers a custom browser launcher. Using this you can launch selenium tests in any browser you like(there will be some problems).

Javascript allows us to do a lot of things on client-side(rollovers, popups, banners etc) that make our life on the internet better, but there will always be a bad side(javascript can be used as malware code). One of the security measure that was taken is that you can't automaticlly upload a file while running selenium tests, while most of the web applications that are tested have upload features. Selenium has a few browser launchers that are avoiding security measures^[3] so you can test upload features and other features that have security problems.

One of this browser launchers is *chrome. The *chrome launcher came with selenium-core-0.9. Important to keep in mind is the fact that this is still a experimental browser launcher, so it has a lot of problems(I'll explain one of the problems I encounter while testing XWiki Workspaces).

How to include this in the TestSuite Java class ?

In XWiki for selenium tests there is a class:XWikiSeleniumTestSetup that extends TestSetup class. In this class are set the parameters for selenium tests on the XWiki application. One of this parameters is the browser launcher that you are gone use(the 3rd one of the DefaultSelenium()).

protected void setUp() throws Exception
{
    	this.selenium = new DefaultSelenium("localhost", SeleniumServer.DEFAULT_PORT, "*chrome /usr/lib/firefox/firefox-2-bin", BASE_URL);

        // Sets the Selenium object in all tests
        for (AbstractXWikiTestCase test: getTests(getTest())) {
            test.setSelenium(this.selenium);
        }

        this.selenium.start();
}

Comparing *firefox launcher with *chrome launcher

If you check the forums you will see that one of the most debated issuse is the Selenium internal error: Security error. Most of the users have encounter this error while trying to test upload feature. Take this code for example:

getSelenium().click("//a[text()='Upload a new file +']");
getSelenium().type("xwikiuploadfile", "/some_path/Documents/1231745.html");
getSelenium().click("//input[@value='Add this attachment']");
getSelenium().waitForPageToLoad("30000");

With *firefox launcher this code was returning Security error. Then I decide to try something: the chrome launcher and it did work(for specifications of using chrome launcher check above at the setUp() method).

Because *chrome launcher is still experimental it has problems. One of the problems that I've encountered was when I tested the next code:

this.typeInWysiwyg("is a wonderful wonderful day selenium test.");

Instead of typing letter y and the rest of the text, the *chrome launcher opened the browser history. I've then tested the same code with *firefox launcher and everything was fine: the string was typed into the field and the browser history wasn't opened. This being a problem because when tests are ran, even that a browser is launched per test, all of the browser instances will be launched by the browser launcher that have been set in XWikiSeleniumTestSetup class(this is the class where you set up the selenium test paramaters - including the setUp() method).

External links

• Official site of the Selenium Framework http://selenium.openqa.org
• Setting up a selenium test environment with maven http://i-proving.ca/space/Technologies/Maven/Maven+Recipes/Integration+Testing+using+Selenium
• The Selenium Maven Plugin http://mojo.codehaus.org/selenium-maven-plugin/