Testing
Contents
Unit Testing
- These are tests performed in isolation using Mock Objects. More specifically we're using JMock in XWiki.
- These tests must not interact with the environment (Database, Container, File System, etc) and do not need any setup to execute
- Name the Test class with the name of the class under test suffixed with Test. For example the JUnit test class for XWikiMessageTool should be named XWikiMessageToolTest
- Name the test methods with the method to test followed by a qualifier describing the test. For example testImportWithHeterogeneousEncodings().
- Using JUnit 4.x and JMock 2.x
- Module tests should depend on the shared-tests module and especially on AbstractComponentTestCase.
Functional Testing
- These are tests done from a running XWiki instance.
XHTML, CSS & WCAG Validations
- We are using JUnit to validate that all XE pages produce valid XHTML.
- Existing tests can be found in enterprise/distribution-test/misc-tests/
- WCAG Testing Strategy
- CSS validation must be verified manually using the W3C validator. See XHTML & CSS Coding Style.
GUI tests
- We are using Selenium RC to perform functional tests for GUI. We have created some JUnit extension to easily write Selenium tests in Java.
- Existing tests can be found in enteprise/distribution-test/selenium-tests/
- To run these tests on your local machine go to enterprise/distribution-test/selenium-tests and type mvn install.
- To run a specific test, pass the pattern property as in: mvn install -Dpattern=DeletePageTest (this will run the DeletePageTest - Note that you don't have to specify the extension). In addition if you wish to execute only a specific method from a Test Case class, you can pass the patternMethod property as in: mvn install -Dpattern=DeletePageTest -DpatternMethod=testDeletePageCanDoRedirect.
- To enable debug output from the selenium server start maven with the -Ddebug=true switch and then all messages from the selenium server are saved to: enterprise/distribution-test/selenium-tests/target/selenium/server.log.
- To debug a functional Selenium test in your favourite Java IDE go in enterprise/distribution-test/selenium-tests and run maven with -Dmaven.surefire.debug. Maven will wait till you connect your IDE to the running JVM on port 5005. You can put a breakpoint in your IDE and debug the test.
Performance Testing
- These are memory leakage tests, load tests and speed of execution tests.
- They are performed manually and in an ad hoc fashion for now.
See the Profiling topic for details on how to use a profiler for detecting performance issues.
Manual testing
Besides automated testing, ensuring software quality requires that features be tested by actual users (see http://en.wikipedia.org/wiki/Manual_testing). In order to manage manual testing, a test plan is required.
Test Coverage
We use Clover to generate test coverage reports.
To generate a Clover report, make sure you can build your module and type mvn site -Pclover. This will generate a Clover report in target/site/clover/index.html.
Here are some sample reports (note that the coverage doesn't include our functional tests for the moment):