Wiki source code of Testing

Last modified by Vincent Massol on 2026/04/04 18:23

Hide last authors
Eduard Moraru 60.2 1 {{box cssClass="floatinginfobox" title="**Contents**"}}
2 {{toc/}}
3 {{/box}}
Vincent Massol 17.2 4
Vincent Massol 93.1 5 There are various types of tests and everyone has its own terminology. You'll find below the XWiki terminology and best practices related to testing.
Vincent Massol 57.1 6
Vincent Massol 93.1 7 Check the [[general development flow>>dev:Community.DevelopmentPractices||anchor="HGeneralDevelopmentFlow"]] to understand better how testing fits in the larger picture.
8
Vincent Massol 138.2 9 = Testing Strategy =
10
Vincent Massol 138.3 11 Here's the general testing methodology used by the XWiki project:
Clément Aubin 95.1 12
Lucas Charpentier 168.1 13 * Code committed must have associated automated tests.
Vincent Massol 143.1 14 * There are [[checks in the build>>Community.Building||anchor="HAutomaticChecks"]] (in the ##quality## profile - executed automatically on the CI) to ensure that committed code:
15 ** do not reduce the total test coverage for that module
Vincent Massol 131.1 16 * Developers run unit and functional tests on their machines daily and frequently.
Vincent Massol 138.4 17 * Unit and functional tests are executed by the CI, at each commit.
Vincent Massol 130.1 18 * Most automated functional tests are currently running in a single environment (HSQLDB/Jetty/Firefox). However, we've started to use Docked-based testing to test automatically using several configurations and are migrating more and more tests to that. There are jobs in XWiki's CI that execute tests in all configurations regularly (not at each commit though but as much as possible).
19 ** Manual tests are also executed at each release by dedicated contributors and they manually execute tests that are not covered by the automated tests.
Vincent Massol 138.5 20 * Performance tests are executed manually several times per year (usually for LTS releases and sometimes for stable releases too).
Vincent Massol 95.2 21 * Responsiveness tests (e.g. verify that XWiki displays fine on a Mobile) are currently executed manually from time to time by dedicated contributors.
Vincent Massol 138.6 22 * Accessibility (WCAG) tests are performed automatically at each commit.
Vincent Massol 95.2 23 * HTML5 validity tests are performed automatically at each commit.
Vincent Massol 93.1 24
Vincent Massol 158.1 25 == Choosing a type of test to write ==
26
27 The strategy is the following:
28
29 * We need at least one functional test for each feature to make sure the feature really works end to end.
30 * We should limit the numer of functional tests we write because:
31 ** They're costly to write
32 ** They're costly to run (takes a long time to setup the XWiki instance and they're executed on various environments too).
33 ** They're costly to maintain (functional tests are more subject to flickers)
Vincent Massol 158.3 34 ** You get feedback about a problem much later on. A unit or integration tests will be executed when building that module, all the time, while functional tests will give you results hours later once the CI execute them.
Vincent Massol 158.1 35 * When testing variations for a use case then we must favor writing a unit test or an integration tests since they're less costly to execute and run. This can mean: a java unit test, a Javascript unit test (using Jasmine for example), a Java integration test, a ##PageTest## (to test a wiki page or a template).
Vincent Massol 158.7 36 * Ask yourself first what would be the fastest executing test to write that would prove that the change you're bringing to the code is tested. Imagine you're making a change in some JS file for example; writing a functional test for that would be bad and not ideal because of the points listed above and it would be much better to have a Jasmine-based unit test. One exception is if the general feature where the change is located doesn't currently have a fuctional test and thus writing a functional test would cover the generic use case and incidentally also cover the change (without being specific to that change).
Vincent Massol 158.1 37
Vincent Massol 166.1 38 == Testing Guidelines ==
39
40 * Don't use sensitive information in test data.
41
Vincent Massol 20.1 42 = Unit Testing =
Guillaume Lerouge 12.1 43
Jerome 44.2 44 == Java Unit Testing ==
45
Vincent Massol 131.2 46 See [[Java Unit Testing>>Community.Testing.JavaUnitTesting.WebHome]]
Vincent Massol 106.2 47
Jerome 44.2 48 == JavaScript Unit Testing ==
49
Simon Urli 162.1 50 {{warning}}
51 The ##jasmine-maven-plugin## we use to run the Javascript test on some module might not work on some distribution, such as Debian buster. A workaround documented as part of https://jira.xwiki.org/browse/XWIKI-17287 is to use {{code}}export OPENSSL_CONF=""{{/code}} to make it work. Also note that we're considering to get rid of Jasmine Maven Plugin in the future since it seems not maintained anymore, you can find information on https://jira.xwiki.org/browse/XWIKI-19316.
52 {{/warning}}
53
Lavinia Vitel 152.1 54 * These are **tests that do not rely on the DOM**, written as "behavioral specifications", using the [[Jasmine>>https://jasmine.github.io/]] test framework. You can find some information how to write such tests on [[the frontend resources>>xwiki:Documentation.DevGuide.FrontendResources.IntegratingJavaScriptLibraries||anchor="HWhatisJasmine3F"]].
Vincent Massol 159.1 55 * In development mode, you can start the test runner process by running {{code language="shell"}}mvn jasmine:bdd{{/code}} in your command line. This will start a test runner at ##http:~/~/localhost:8234##, that will run all tests in the ##src/test/javascript## directory. Write your tests there and hit refresh to see the results directly in your browser.
Manuel Smeria 64.5 56 * For tests that need a DOM, see [[Functional Testing>>||anchor="HFunctionalTesting"]]
Jerome 44.2 57
Vincent Massol 57.1 58 == Java Rendering Testing ==
59
Manuel Smeria 64.5 60 We have a special framework for making it easy to write Rendering tests, see the [[Rendering Testing Framework>>rendering:Main.Extending||anchor="HAddingTests"]]
Vincent Massol 57.1 61
Vincent Massol 159.2 62 == View Unit Testing ==
Vincent Massol 80.1 63
Vincent Massol 159.2 64 See [[View Unit Testing>>Community.Testing.ViewUnitTesting.WebHome]]
Vincent Massol 80.1 65
Vincent Massol 57.1 66 = Functional Testing =
67
68 A functional test requires a running XWiki instance.
69
Vincent Massol 138.7 70 == Functional UI Testing ==
Vincent Massol 20.1 71
Vincent Massol 170.2 72 We now have 3 frameworks for running UI tests:
Eduard Moraru 60.2 73
Vincent Massol 170.2 74 * One based on Docker (and using Selenium 4+). This is the recommended framework to use for new tests.
Vincent Massol 160.1 75 * One based on Selenium3/Webdriver which is now deprecated and shouldn't be used. We encourage committers to port tests written for it to Docker tests. Especially when committers bring modification to the old tests we encourage them to rewrite the tests as new Docker tests.
Vincent Massol 140.2 76 * A last one based on Selenium1 which is also deprecated and shouldn't be used. We encourage committers to port tests written for it to Docker tests. Especially when committers bring modification to the old tests we encourage them to rewrite the tests as new Docker tests.
Vincent Massol 31.1 77
Vincent Massol 138.7 78 {{id name="HSelenium3-basedFramework"/}}
Vincent Massol 114.1 79
Vincent Massol 138.7 80 === Docker-based Testing ===
Eduard Moraru 128.1 81
Vincent Massol 138.7 82 This is currently the official way to write UI functional tests.
Vincent Massol 114.1 83
Vincent Massol 138.7 84 See [[DockerTesting]].
Vincent Massol 117.1 85
Vincent Massol 171.1 86 === Best Practices ===
Vincent Massol 31.1 87
Vincent Massol 171.1 88 * Check the best practices defined in the target functional framework used, for example in [[DockerTesting]] for docker-based tests.
Vincent Massol 101.1 89 * Tests are located in ##xwiki-platform## inside the module that they are testing. Note that in the past we were putting functional tests in ##[[xwiki-platform-distribution/xwiki-platform-distribution-flavor/xwiki-platform-distribution-flavor-test>>https://github.com/xwiki/xwiki-platform/tree/master/xwiki-platform-distribution/xwiki-platform-distribution-flavor/xwiki-platform-distribution-flavor-test]]## but we have started to move them inside the specific modules they are testing.
90 * Name the tests using ##<prefix>IT.java##.
Vincent Massol 171.1 91 * Use a ##AllIT.java## test suite to ensure that XWiki is started/stopped only once for all tests in the module and use nested tests. For example:(((
Vincent Massol 101.1 92 {{code language="java"}}
Vincent Massol 171.1 93 @UITest
Vincent Massol 163.3 94 public class AllIT
Vincent Massol 101.1 95 {
Vincent Massol 171.1 96 @Nested
97 class NestedMentionsIT extends MentionsIT
98 {
99 }
Vincent Massol 101.1 100 }
101 {{/code}}
102 )))
103 * Use the ##maven-failsafe-plugin## for functional UI tests:(((
104 {{code language="xml"}}
105 <plugin>
106 <groupId>org.apache.maven.plugins</groupId>
107 <artifactId>maven-failsafe-plugin</artifactId>
108 </plugin>
109 {{/code}}
110 )))
111 * Locate tests in ##src/test/java## (i.e the default maven location for tests)
Vincent Massol 81.2 112 * Tests should be written using the Page Objects Pattern:
113 ** [[Original article from Simon Steward>>https://code.google.com/p/selenium/wiki/PageObjects]]
114 ** [[Article from Martin Fowler>>http://martinfowler.com/bliki/PageObject.html]]
Vincent Massol 171.1 115 * Since functional tests take a long time to execute (XWiki to start, Browser to start, navigation, etc) it's important to write tests that execute as fast as possible. Here are some tips:
116 ** Write scenarios (i.e. write only a few test methods, even only one if you can so that you can have a single fixture) instead of a lot of individual tests as you'd do for unit tests.
117 ** Use ##TestUtils## calls to perform quick actions that are not part of what you wish to test (i.e that are part of the test fixture). For example to create a page you need in your fixture, write the following and DO NOT navigate likea user would do using the XWiki UI:(((
Vincent Massol 56.1 118 {{code language="java"}}
Vincent Massol 171.1 119 @Test
120 void guestUser(TestUtils testUtils, TestReference testReference)
121 {
122 // Test fixture
123 testUtils.loginAsSuperAdmin();
124 testUtils.createPage(testReference, "some content");
125 ...
126 }
Vincent Massol 56.1 127 {{/code}}
Lavinia Vitel 152.1 128 )))
Simon Urli 148.1 129 * Your Maven module for test may depend on a specific profile (as a best practice, you can use ##integration-tests##). Doing this, it will be built only when specifically asked with ##-Pintegration-tests## (see below for an example of the ##pom.xml##)(((
Jean SIMARD 80.3 130 {{code language="xml"}}
131 <profiles>
132 <profile>
133 <id>integration-tests</id>
134 <modules>
135 <module>application-task-test</module>
136 </modules>
137 </profile>
138 </profiles>
139 {{/code}}
140 )))
Simon Urli 148.1 141 )))
Manuel Smeria 64.5 142 * {{warning}}Never, ever, wait on a timer!{{/warning}} Instead wait on elements.
Vincent Massol 101.1 143 * Tests that need to change existing configuration (e.g. change the default language, set specific rights, etc) must put back the configuration as it was. This is true only in flavor tests or when several functional tests of different domains are executed one after another. However functional tests located in xwiki-platform specific modules are only running the tests for their module and thus it's not important, and saves times, if they don't clean up.
Manuel Smeria 64.5 144 * Tests are allowed to create new documents and don't need to remove them at the end of the test.
Simon Urli 148.1 145
Vincent Massol 72.1 146 Examples of functional tests:
Eduard Moraru 75.1 147
Vincent Massol 72.1 148 * {{scm path="xwiki-platform-core/xwiki-platform-activeinstalls/xwiki-platform-activeinstalls-test"}}Active Installs Application Functional Tests{{/scm}}
149 * {{scm path="xwiki-platform-core/xwiki-platform-administration/xwiki-platform-administration-test"}}Administration Application Functional Tests{{/scm}}
150 * {{scm path="xwiki-platform-core/xwiki-platform-linkchecker/xwiki-platform-linkchecker-test"}}Link Checker Application Functional Tests{{/scm}}
151 * {{scm path="xwiki-platform-core/xwiki-platform-panels/xwiki-platform-panels-test"}}Panels Application Functional Tests{{/scm}}
152
Vincent Massol 171.1 153 === Legacy Frameworks ===
154
155 In the history of XWiki's development we have built different frameworks to perform functionl tests and we still haven't finished migrating them to the newer frameworks. This is an ongoing process. When fully done, these instructions should be removed.
156
157 ==== Selenium-based Framework ====
158
159 Using:
160
161 * Uses [[Selenium3>>https://www.selenium.dev/]]
162 * To debug a test simply start XWiki somewhere and then debug your JUnit tests as a normal JUnit test in your IDE.
163 ** Note that functional test Maven modules will create an XWiki instance in ##target/xwiki## thanks to the execution of the XWiki Packager plugin, so it's simpler to start this XWiki instance when debugging.
164 * In order to debug more easily flickering tests you can simply add the {{{@Intermittent}}} annotation on your test method and it'll be executed 100 times in a row (you can also specify {{{@Intermittent(repetition=N)}}}to repeat it N times). This is achieved thanks to the [[Tempus Fugit>>http://tempusfugitlibrary.org/documentation/junit/intermittent/]] framework that we've integrated.
165 * To run a specific test, pass the ##pattern## property (it's a regex on the test class name) as in: ##mvn install -Dpattern="TestClass#testName"## (this will run the ##testName## test from ##TestClass##)
166 ** More details on the usage of the ##pattern## property and examples can be found in the [[documentation of the XWikiExecutorTestMethodFilter class>>https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwiki-platform-test/xwiki-platform-test-integration/src/main/java/org/xwiki/test/integration/XWikiExecutorTestMethodFilter.java#L25]].
167 * To run the tests on your own running instance (instead of letting Maven close your instance and start a fresh one), use ##-Dxwiki.test.verifyRunningXWikiAtStart=true##. It could be useful to verify that you have not broken the tests on your instance before committing your changes.
168 * By default the Firefox browser will be used but if you wish to run with another browser just pass the ##browser## parameter as in:
169 ** Firefox (default): ##-Dbrowser=*firefox##
170 ** Internet Explorer: ##-Dbrowser=*iexplore##
171 ** Chrome: ##-Dbrowser=*chrome##
172 ** PhantomJS: ##-Dbrowser=*phantomjs##
173 * In case of compatibility problem between the version of the browser and the version of Selenium, you can specify which install of firefox to use thanks to ##-Dwebdriver.firefox.bin##. For example, we use to need Firefox 32 in the past, you could install it locally and refer to it with ##-Dwebdriver.firefox.bin="/path/to/your/firefox-32.0"##.
174 * We check for pages that require Programming Rights automatically by bundling a Listener component that listens to the ##ScriptEvaluatingEvent## event and that drops Programming Rights, in an effort to make the tests fail and so that the developer can notice he requires PR and fix that. In some cases where it's necessary, this can be disabled by setting the configuration {{code language="xml"}}< testProgrammingRights>false</testProgrammingRights>{{/code}} in the configuration of the Package Mojo, and/or by using a System Property to control where the check is performed, e.g. {{code language="xml"}}<xwikiPropertiesAdditionalProperties>test.prchecker.excludePattern=.*:XWiki\.XWikiPreferences</xwikiPropertiesAdditionalProperties>{{/code}}. {{info}}Since XWiki 9.8RC1{{/info}}
175 * If the ##xwiki.test.startXWiki## system property is set to ##true## then the test itself will start/stop XWiki. If set to ##false## then it's then the responsibility of the build to start/stop XWiki. Useful when starting.stopping XWiki inside a Docker container handled by the Maven build for example. {{info}}Since XWiki 10.0{{/info}}
176
Vincent Massol 161.1 177 ===== Troubleshooting =====
178
Lucas Charpentier 168.1 179 * If you get an error of the type ###java.net.ConnectException: Failed to connect to localhost/0:0:0:0:0:0:0:1:21684#, try removing the ##WebDriverManager## cache directory at ##/.cache/selenium## and retry.##
Vincent Massol 161.1 180
Vincent Massol 153.1 181 ==== Selenium1-based Framework ====
Vincent Massol 31.1 182
183 * We were using Selenium RC to perform functional tests for GUI. We had created some JUnit extension to easily write Selenium tests in Java.
Eduard Moraru 60.2 184 ** Existing tests can be found in ##[[xwiki-enteprise/xwiki-enterprise-test/xwiki-enterprise-test-selenium>>https://github.com/xwiki/xwiki-enterprise/tree/master/xwiki-enterprise-test/xwiki-enterprise-test-selenium]]##
185 * To run these tests on your local machine go to ##xwiki-enteprise/xwiki-enterprise-test/xwiki-enterprise-test-selenium## and type ##mvn install##.
Guillaume Lerouge 13.1 186 * To run a specific test, pass the ##pattern## property as in: ##mvn install -Dpattern=DeletePageTest## (this will run the ##DeletePageTest## - Note that you don't have to specify the extension). In addition if you wish to execute only a specific method from a Test Case class, you can pass the ##patternMethod## property as in: ##mvn install -Dpattern=DeletePageTest -DpatternMethod=testDeletePageCanDoRedirect##.
Eduard Moraru 60.2 187 * To enable debug output from the selenium server start maven with the ##-Ddebug=true## switch and then all messages from the selenium server are saved to: ##xwiki-enteprise/xwiki-enterprise-test/xwiki-enterprise-test-selenium/target/selenium/server.log##.
188 * To debug a functional Selenium test in your favourite Java IDE go in ##xwiki-enteprise/xwiki-enterprise-test/xwiki-enterprise-test-selenium## and run maven with ##-Dmaven.surefire.debug##. Maven will wait till you connect your IDE to the running JVM on port 5005. You can put a breakpoint in your IDE and debug the test.
Vincent Massol 10.2 189
Vincent Massol 153.1 190 ==== Browser version ====
Eduard Moraru 73.1 191
Vincent Massol 153.1 192 The legacy frameworks only supported testing with one version of the ##Firefox## browser.
Eduard Moraru 73.1 193
Vincent Massol 157.1 194 * The ##Selenium 2## version used is ##2.44.0##
Eduard Moraru 77.2 195 ** (valid for 18.March.2015. Actual version used can be verified [[here>>https://github.com/xwiki/xwiki-commons/blob/master/pom.xml]] under the ##selenium.version## property)
Vincent Massol 153.1 196 * The ##Firefox## version we use in you continuous integration agents can be [[found in our docker build image>>https://github.com/xwiki/xwiki-docker-build/blob/master/build/Dockerfile#L55]].
Eduard Moraru 77.2 197 ** To determine browser compatibility with the Selenium version used, scan [[Selenium's changelog>>https://code.google.com/p/selenium/source/browse/java/CHANGELOG]] and look for entries like "* Updating Native events to support Firefox 24, 31, 32 and 33". That shows the supported browser version for the particular Selenium version.
Eduard Moraru 73.1 198
Eduard Moraru 157.2 199 If you wish to run tests with the exact configuration as XWiki's [[Continous Integration server>>http://ci.xwiki.org]] uses, you should use the [[XWiki Build Docker Image>>Community.Building.WebHome||anchor="HBuildingwithDocker"]]. This will get you the correct Firefox version pre-installed.
Eduard Moraru 75.1 200
Vincent Massol 157.1 201 If you wish to do that manually without Docker then you have to:
202
Eduard Moraru 73.1 203 1. [[Download>>https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/]] the corresponding Firefox release
204 1. Unzip it locally
205 1. Use the ##webdriver.firefox.bin## java system property to specify the location of your firefox version
Vincent Massol 74.2 206 11. Depending on how you are starting the functional tests, you`d have to either add the system property in your maven build (surefire plugin configuration) or in your IDE (run configuration)
207 11. Read [[Selenium's FirefoxDriver documentation>>https://code.google.com/p/selenium/wiki/FirefoxDriver]] for more information and options
Eduard Moraru 73.1 208
Vincent Massol 153.1 209 ==== Adding a new maven dependency for minimal war ====
Simon Urli 147.1 210
211 All our functional testing are linked to the build of a custom instance of XWiki based on the dependencies contained in the pom.xml of the test.
212 However, those dependencies are added to a minimal war that is built using dependencies located on different places dependending if it is a docker test, or an older functional test.
213
214 So when a new Maven dependency is added for all distributions, it should be declared in three places:
215
Vincent Massol 158.8 216 * in ##xwiki-platform-minimaldependencies/pom.xml##: this will allow the dependency to be retrieved for both docker tests, and for the standard distribution
217 * in ##xwiki-tools-packager-plugin##, in both ##pom.xml## and ##PackageMojo.java##: those are used for older functional tests.
Simon Urli 148.1 218
Vincent Massol 150.1 219 == IDE Support ==
220
Eduard Moraru 157.3 221 Functional tests are JUnit tests like unit tests. Thus you can execute them from within your IDE. However we usually have them enabled only when the [[##integration-tests## and ##docker## Maven profiles>>Community.Building.WebHome||anchor="HUsingProfiles"]] are enabled so make sure to enable these profiles in your IDE.
Vincent Massol 150.1 222
223 For example with IntelliJ IDEA:
224
225 {{image reference="intellijideafunctionaltest.png" width="650px"/}}
226
Vincent Massol 154.1 227 == Maven ==
228
Vincent Massol 156.1 229 * You can automatically start the XWiki instances in debug mode (i.e. the XWiki JVM will be configured to wait for a client to connect to the debugging port), by passing the ##-Ddebug=true## system property ,as in:(((
Eduard Moraru 157.2 230 {{code language="sh"}}
Vincent Massol 154.1 231 mvn clean verify -Ddebug=true
232 {{/code}}
233
Eduard Moraru 157.2 234 See the [[Remote Debugging page>>Community.Debugging||anchor="HRemoteDebugging"]] for more details.
Vincent Massol 156.1 235 )))
236 * You can control the XWiki start timeout from the command line with the ##xwikiExecutionStartTimeout## system property, for example:(((
Eduard Moraru 157.2 237 {{code language="sh"}}
Vincent Massol 156.1 238 mvn clean verify -Ddebug=true -Dpattern=.*documentModifiedCacheSync.* -DxwikiExecutionStartTimeout=1200
239 {{/code}}
240 )))
Vincent Massol 155.1 241
Vincent Massol 91.1 242 = XHTML, CSS & WCAG Validations =
Vincent Massol 57.1 243
Vincent Massol 163.1 244 * We are using JUnit to validate that all XWiki pages produce valid XHTML 5 content.
Lucas Charpentier 165.1 245 https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-distribution/xwiki-platform-distribution-flavor/xwiki-platform-distribution-flavor-test/xwiki-platform-distribution-flavor-test-webstandards/src/test/it/org/xwiki/test/webstandards/AllTests.java
Vincent Massol 163.1 246 ** Existing tests can be found in {{scm path="xwiki-platform-distribution/xwiki-platform-distribution-flavor/xwiki-platform-distribution-flavor-test/xwiki-platform-distribution-flavor-test-webstandards"}}##xwiki-platform-distribution-flavor-test-webstandards##{{/scm}}
Manuel Smeria 64.5 247 * [[WCAG Testing Strategy>>WCAGTesting]]
Vincent Massol 163.1 248 * RSS validation is also done in {{scm path="xwiki-platform-distribution/xwiki-platform-distribution-flavor/xwiki-platform-distribution-flavor-test/xwiki-platform-distribution-flavor-test-webstandards"}}##xwiki-platform-distribution-flavor-test-webstandards##{{/scm}}
Manuel Smeria 64.5 249 * CSS validation must be verified manually using the [[W3C validator>>http://jigsaw.w3.org/css-validator/]]. See [[XHTML & CSS Coding Style>>XhtmlCssCodeStyle]].
Vincent Massol 57.1 250
Vincent Massol 20.1 251 = Performance Testing =
VincentMassol 1.2 252
VincentMassol 1.1 253 * These are memory leakage tests, load tests and speed of execution tests.
Vincent Massol 92.2 254 * They are performed manually and in an ad hoc fashion for now. They are executed for some stable versions and for all super stable versions.
255 * See [[Methodology and reports>>test:Performances.WebHome]].
Guillaume Lerouge 14.1 256
Guillaume Lerouge 12.1 257 See the [[Profiling topic>>Profiling]] for details on how to use a profiler for detecting performance issues.
VincentMassol 1.1 258
Vincent Massol 20.1 259 = Manual testing =
VincentMassol 1.22 260
Eduard Moraru 60.2 261 {{info}}
Manuel Smeria 78.1 262 Here's the spirit we'd like to have from the XWiki Manual Testing Team: [[The Black Team>>http://www.t3.org/tangledwebs/07/tw0706.html]].
Eduard Moraru 60.2 263 {{/info}}
Vincent Massol 43.1 264
Manuel Smeria 64.5 265 Besides automated testing, ensuring software quality requires that features be tested by actual users (see [[Manual testing>>http://en.wikipedia.org/wiki/Manual_testing]]). In order to manage manual testing, a [[test plan>>http://en.wikipedia.org/wiki/Test_plan]] is required.
Guillaume Lerouge 11.1 266
Vincent Massol 94.1 267 * [[Manual test reports>>test:Main.WebHome]]
Manuel Smeria 64.5 268 * [[Manual Test Strategy>>ManualTestStrategy]]
Sorin Burjan 49.1 269 * [[Manual Test Reports>>xwiki:TestReports.WebHome]]
Guillaume Lerouge 11.1 270
Vincent Massol 169.1 271 = Security Testing =
272
273 Security issues are detected by a combination of:
274 * Code reviews done by committers and the community
275 * Security detection tools that are executed in our CI or on our GitHub repositories
Vincent Massol 170.1 276 * There are {{scm path="xwiki-platform-distribution/xwiki-platform-distribution-flavor/xwiki-platform-distribution-flavor-test/xwiki-platform-distribution-flavor-test-escaping"}}automated escaping test{{/scm}} that execute to try to find possible injections
Vincent Massol 169.1 277
278 See [["How we detect security issues">>dev:Community.SecurityPolicy.WebHome||anchor="HHowwedetectnewsecurityissues3F"]] for more details.
279
Vincent Massol 92.1 280 = Tools Reference =
281
282 We use the following tools in our automated tests:
Clément Aubin 95.1 283
Vincent Massol 132.1 284 * [[JUnit>>https://junit.org/junit5/]]: test framework
285 * [[Mockito>>https://site.mockito.org/]]: mocking environment of unit test to isolate it
286 * [[Hamcrest>>http://hamcrest.org/JavaHamcrest/]]: extra assertions beyond what JUnit provides
Vincent Massol 92.1 287 * [[GreenMail>>http://www.icegreen.com/greenmail/]]: for testing email sending
288 * [[WireMock>>http://wiremock.org/]]: for simulating HTTP connections to remote servers
Vincent Massol 138.1 289 * [[Selenium>>https://seleniumhq.github.io/docs/]]: for functional UI test to simulate a user interacting with XWiki
290 * [[TestContainers>>https://www.testcontainers.org/]]: Docker-based functional testing
Vincent Massol 92.1 291 * [[JMeter>>http://jmeter.apache.org/]]: for performance tests
292 * [[Dumbbench>>https://github.com/tsee/dumbbench]]: for manual performance tests
293
Vincent Massol 20.1 294 = Test Coverage =
Guillaume Lerouge 11.1 295
Vincent Massol 133.1 296 See the [[TestCoverage]] page.

Get Connected