Show last authors
1 {{box cssClass="floatinginfobox" title="**Contents**"}}
2 {{toc/}}
3 {{/box}}
4
5 = General =
6
7 XWiki has a [[Continuous Integration>>http://www.martinfowler.com/articles/continuousIntegration.html]] setup to ensure XWiki's code is built at all times (i.e at every code check in). This also allows developers to only build the module they want and they'll have the other XWiki module dependencies downloaded from the [[XWiki remote repository>>http://nexus.xwiki.org/nexus/#view-repositories]].
8
9 We use the following tools:
10
11 * [[GitHub>>https://github.com/xwiki]] is used as our [[SCM>>http://en.wikipedia.org/wiki/Source_Code_Management]]
12 * [[Maven>>http://maven.apache.org/]] is used for [[building XWiki>>Community.Building]]
13 * [[Jenkins>>http://jenkins-ci.org/]] is used as our Continuous Integration (CI) tool (set up at [[http://ci.xwiki.org/]])
14 * [[Docker>>https://www.docker.com/docker-community]] is used to run Jenkins agents (and incidentally to execute some functional tests over various configurations)
15
16 We use a technique called [[binary dependency build>>http://web.archive.org/web/20090423073100/http://blogs.codehaus.org/people/vmassol/archives/000953_binary_dependency_builds.html]] which allows to check out only the module a developer wishes to work on and he'll automatically get the latest fresh binary dependencies built by our CI tool.
17
18 = CI Requirements =
19
20 We're currently using Jenkins as our CI tool but it's possible to change. We're listing here our requirements for a CI tool, and with some of the solutions we've evaluated or are still evaluating.
21
22 == REQ1: Parallel Execution ==
23
24 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
25 |REQ1|Ability to run parallel and serial builds, with different parameters|(/)|(/)|(/)|(/)
26
27 * Jenkins: Using the ##parallel()## step
28 * GitLab CI: Using [[##trigger##, ##rules## and ##needs##>>https://docs.gitlab.com/ee/ci/pipelines/pipeline_architectures.html]]. See also [[multi-project triggering>>https://docs.gitlab.com/ee/ci/multi_project_pipelines.html]]. See also [[Child pipelines>>https://docs.gitlab.com/ee/ci/parent_child_pipelines.html]]. General doc for [[pipeline types>>https://docs.gitlab.com/ee/ci/pipelines/#types-of-pipelines]].
29 * GitHub Actions: Using different YML files in ##.github/workflows## (jobs run in parallel by default) and using [[##job.needs##>>https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobs]].
30 * CircleCI: See [[https://circleci.com/blog/decrease-your-build-times-by-running-jobs-in-parallel-with-workflows/]]
31
32 == REQ2: Scheduling ==
33
34 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
35 |REQ2|Ability to schedule builds (e.g. for the various docker tests) and to run the build at least once per month even if there's been no changes to the source|(!)|(!)|(/)|(/)
36
37 * Jenkins: Scheduling is supported but it's [[very complex to make it work automatically with a pipeline>>https://massol.myxwiki.org/xwiki/bin/view/Blog/ScheduledJenkinsfile]] since a repo can have only a single pipeline (i.e. a single ##Jenkinsfile##). Other solutions include using a Git submodule or another GitHub Organization Job.
38 * GitLab CI: It's possible to have [[schedules>>https://docs.gitlab.com/ee/ci/pipelines/schedules.html]] but note that the schedule is not part of the YML file but something you define in the GitLab CI configuration UI. However, it's also possible to [[use the API>>https://docs.gitlab.com/ee/api/pipeline_schedules.html#create-a-new-pipeline-schedule]].
39 * Github Actions: Using the [[##on## element with a ##schedule##>>https://help.github.com/en/actions/reference/events-that-trigger-workflows#scheduled-events-schedule]].
40 * CircleCI: See https://circleci.com/docs/2.0/configuration-reference/#schedule
41
42 == REQ3: Sharing Pipeline Code ==
43
44 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
45 |REQ3|Ability to share pipeline code between builds for the same repositoryy and for different repositories. And ability to pass properties to this common code.|(/)|(/)|(/)|(/)
46
47 * Jenkins: Using a shared pipeline
48 * GitLab CI: Using an [[##include##>>https://docs.gitlab.com/ee/ci/yaml/README.html#include]].
49 * GitHub Actions: With a [[custom action>>https://help.github.com/en/actions/building-actions]] at least (feels a bit complex).
50
51 == REQ4: View Test Results ==
52
53 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
54 |REQ4|Ability to view test results and failing test logs (test logs + console/xwiki logs).|(/)|(x)|(!)|(?)
55
56 * Jenkins: Built in.
57 * GitLab CI: Built in but [[not working ATM because of performance issues>>https://gitlab.com/groups/gitlab-org/-/epics/2854]].
58 * GitHub Actions: Needs a custom action ATM. Found 2 but didn't work when I tested them: [[check run reporter>>https://www.check-run-reporter.com/]] ([[problem reported>>https://github.com/check-run-reporter/feedback/issues/3]]) and [[Scope>>https://scope.dev/]] (2 problems: unknown cost + got stuck when I tested it on commons and I had to kill the workflow after 1 hour).
59 ** Update: [[new ScaCap surefire report action>>https://github.com/ScaCap/action-surefire-report]] to test.
60
61 == REQ5: Disable Concurrent Builds ==
62
63 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
64 |REQ5|(optional) Ability to aggregate commits before a build is executed, i.e. disable concurrent builds. Note that this is more a limitation of the number of agents we can have vs the duration of the full xwiki build. If we get a large number of agents, it may be a less important requirement (or not one at all).|(/)|(?)|?/)|(/)
65
66 * Jenkins: Using the
67 * GitLab: (?)
68 * GitHubActions: (?)
69 * CircleCI: In the Project CI's configuration. It's called "Auto-cancel redundant builds".
70
71 == REQ6: Custom Docker Image ==
72
73 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
74 |REQ6|Supports using a custom docker build image since [[XWiki has one>>https://github.com/xwiki/xwiki-docker-build/tree/master/build]]|(/)|(/)|(/)|(/)
75
76 * Jenkins: Using the Docker Cloud plugin.
77 * GitLab CI: Built in using the ##image## YML element.
78 * GitHub Actions: Buimt in using the [[#container##>>https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idcontainer]] YML element.
79
80 Notes:
81 * That this requirement allows use to control the Java, Maven and more generally our full build environment.
82 * With Jenkins and GitHub Actions the docker daemon socket is shared allowing our docker tests to work. Needs to be checked for GitLab CI.
83
84 == REQ7: Parse pom.xml ==
85
86 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
87 |REQ7|Ability to easily read/parse the current pom.xml to automatically find the version of Java to use and the memory parameters to set for the JVM.|(/)|(?)|(?)|(?)
88
89 * Jenkins: Using the ##readMavenPom## step.
90
91 == REQ8: Shell Scripts ==
92
93 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
94 |REQ8|Ability to execute shell scripts before/after the build, inside our custom build image. For example to execute VNC and export the DISPLAY (for the functional tests not based on Docker)|(/)|(/)|(/)|(/)
95
96 * Jenkins: In the Jenkinsfile/shared pipeline
97 * GitLab CI: Built in, using the ##script## step.
98 * GitHub Actions: Built in, using the ##run## step.
99
100 == REQ9: Archive Artifacts ==
101
102 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
103 |REQ9|Ability to archive artifacts and ability to save the failing test screenshots and videos as artifacts.|(/)|(/)|(/)|(/)
104
105 * Jenkins: Feature of the Maven plugin + ##archiveArtifacts## step.
106 * GitLab CI: Using the ##artifacts## step.
107 * GitHub Actions: Using the [[##upload-artifact## custom action>>https://help.github.com/en/actions/configuring-and-managing-workflows/persisting-workflow-data-using-artifacts]].
108
109 == REQ10: Build Parameters ==
110
111 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
112 |REQ10|Ability to have different build parameters (different maven parameters for example) between master and branches/PR. For example, we want to run "mvn deploy" on master and "mvn install" on branches.|(/)|(/)|(/)|(?)
113
114 * Jenkins: Using method parameters in custom pipeline steps.
115 * GitLab CI: Using [[##variables##>>https://docs.gitlab.com/ee/ci/variables/]]
116 * GitHub Actions: Parameters of a custom action.
117
118 == REQ11: Supports GitHub ==
119
120 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
121 |REQ11|Must work with sources located on GitHub.|(/)|(/)|(/)|(/)
122
123 * Jenkins: GitHub webhook.
124 * GitLab CI: Automatically [[creates a mirrored GitHub repo>>https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/github_integration.html]].
125 * GitHub Actions: By design ;)
126
127 == REQ12: Build PRs ==
128
129 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
130 |REQ12|Ability to run the build on PRs.|(/)|(/)|(/)|(/)
131
132 == REQ13: Timeout ==
133
134 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
135 |REQ13|Ability to kill builds after a given timeout.|(/)|(/)|(?)|(?)
136
137 * Jenkins: Using the ##timeout## step.
138 * GitLab CI: Using the [[##timeout## step>>https://docs.gitlab.com/ee/ci/yaml/#timeout]].
139 * GitHub Actions: (?)
140
141 == REQ14: Secrets ==
142
143 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
144 |REQ14|Handle secrets so that they are not in the pipeline scripts (e.g. we need to push to sonarcloud or deploy to nexus.xwiki.org or to maven central).|(/)|(/)|(/)|(?)
145
146 == REQ15: Mail ==
147
148 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
149 |REQ15|Ability to send mail and customize the mail template.|(/)|(?)|(?)|?)
150
151 * Jenkins: Using the ##emailext## step from the Extended Mail plugin.
152 * GitLab CI: (?)
153 * GitHub Actions: (?)
154
155 == REQ16: Test Screenshots ==
156
157 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
158 |REQ16|(Nice to have) Ability to attach failing test screenshot to failing test report pages|(/)|(?)|(?)|(?)
159
160 == REQ17: Handle Flickers ==
161
162 |=(% width=100px; %)Id|=Requirement|=(% width=100px; %)Jenkins|=(% width=100px; %)GitLab CI|=(% width=100px; %)GitHub Actions|=(% width=100px; %)CircleCI
163 |REQ17|Ability to check failing test (i.e. have an API to retrieve failing tests) so that we can compare with known flickers on our JIRA and decide to send a failing mail or not + mark the test as flickering in the CI Test Report UI.|(/)|(?)|(?)|(?)
164
165 = Jenkins Agent Image =
166
167 XWiki has its own [[Jenkins Agent Docker image>>https://github.com/xwiki/xwiki-jenkins-slave]] that is used by Jenkins master to spawn agents.
168
169 = Jenkins Pipelines =
170
171 == Main Pipeline ==
172
173 We have developed a {{scm project="xwiki-jenkins-pipeline" path="vars/xwikiModule.groovy"}}Jenkins Pipeline script{{/scm}} to build XWiki projects (can be used for core and contrib projects). It has the following features:
174
175 * Automatically use the right version of Java (and proper memory configuration)
176 * Use Jenkins's Xvnc plugin to have a Display for functional tests
177 * Use the "deploy" maven goal for "master" and "stable-*" branches only and "install" for the rest
178 * [[Attach the screenshot>>http://massol.myxwiki.org/xwiki/bin/view/Blog/AttachFailingTestPipeline]] of a failing XWiki Selenium test to the failed test's description
179 * Check for false positives for known cases of failures not related to code + [[check for test flickers>>http://massol.myxwiki.org/xwiki/bin/view/Blog/FlakyTestTool]]
180 * Send mails for build failures
181 * Supports passing custom job properties. See below for an example of how to use this to trigger nightly build of Docker-based configuration tests.
182
183 Note that if you use a "Github Organization" job type in Jenkins you'll get branch management for free, i.e. automatically build branches when new branches are added (or Pull Requests), and automatically stop building branches when they are removed from the SCM.
184
185 To use is for a project, add a ##Jenkinsfile## with the following minimal configuration:
186
187 {{code language="groovy"}}
188 /*
189 * See the NOTICE file distributed with this work for additional
190 * information regarding copyright ownership.
191 *
192 * This is free software; you can redistribute it and/or modify it
193 * under the terms of the GNU Lesser General Public License as
194 * published by the Free Software Foundation; either version 2.1 of
195 * the License, or (at your option) any later version.
196 *
197 * This software is distributed in the hope that it will be useful,
198 * but WITHOUT ANY WARRANTY; without even the implied warranty of
199 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
200 * Lesser General Public License for more details.
201 *
202 * You should have received a copy of the GNU Lesser General Public
203 * License along with this software; if not, write to the Free
204 * Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
205 * 02110-1301 USA, or see the FSF site: http://www.fsf.org.
206 */
207
208 // It's assumed that Jenkins has been configured to implicitly load the vars/xwikiModule.groovy library which exposes
209 // the "xwikiModule" global function/DSL.
210 // Note that the version used is the one defined in Jenkins but it can be overridden as follows:
211 // @Library("[email protected]<branch, tag, sha1>") _
212 // See https://github.com/jenkinsci/workflow-cps-global-lib-plugin for details.
213
214 xwikiModule {
215 }
216 {{/code}}
217
218 For more elaborate configuration, see:
219 * {{scm project="xwiki-commons" path="Jenkinsfile"}}xwiki-commons's Jenkinsfile{{/scm}}
220 * {{scm project="xwiki-platform" path="Jenkinsfile"}}xwiki-platform's Jenkinsfile{{/scm}}
221
222 === Nightly Configuration Tests ===
223
224 Heres an example of how you can setup your ##Jenkinsfile## to automatically have execution of Docker-based configuration tests every night. This example is taken from {{scm project="xwiki-platform" path="Jenkinsfile"}}xwiki-platform's Jenkinsfile{{/scm}}. For more details see [[this blog post>>http://massol.myxwiki.org/xwiki/bin/view/Blog/ScheduledJenkinsfile]].
225
226 * Step 1: In the ##xwikiBuild## step, pass the ##jobProperties## configuration with some scheduler cron. For example:(((
227 {{code language="groovy"}}
228 xwikiBuild(map.name) {
229 ...
230 jobProperties = getCustomJobProperties()
231 ...
232 }
233
234 private def getCustomJobProperties()
235 {
236 // Define a scheduler job to execute the Docker-based functional tests at regular intervals. We do this since they
237 // take time to execute and thus we cannot run them all the time.
238 // This scheduler job will pass the "type" parameter to this Jenkinsfile when it executes, allowing us to decide if
239 // we run the standard builds or the docker ones.
240 // Note: it's the xwikiBuild() calls from the standard builds that will set the jobProperties and thus set up the
241 // job parameter + the crons. It would be better to set the properties directly in this Jenkinsfile but we haven't
242 // found a way to merge properties and calling the properties() step will override any pre-existing properties.
243 return [
244 parameters([string(defaultValue: 'standard', description: 'Job type', name: 'type')]),
245 pipelineTriggers([
246 parameterizedCron('''@midnight %type=docker-latest
247 @weekly %type=docker-all
248 @monthly %type=docker-unsupported'''),
249 cron("@monthly")
250 ])
251 ]
252 }
253 {{/code}}
254 )))
255 * Step 2: Based on the parametrized ##type## value, decide to execute Docker-based tests:(((
256 {{code language="groovy"}}
257 ...
258 if (params.type && params.type == 'docker-latest') {
259 buildDocker('docker-latest')
260 }
261 ...
262 private void buildDocker(type)
263 {
264 def dockerConfigurationList
265 def dockerModuleList
266 def customJobProperties
267 node() {
268 // Checkout platform to find all docker configurations and test modules so that we can then parallelize executions
269 // of configs and modules across Jenkins agents.
270 checkout skipChangeLog: true, scm: scm
271 dockerConfigurationList = dockerConfigurations(type)
272 if (type == 'docker-unsupported') {
273 dockerModuleList = ['xwiki-platform-core/xwiki-platform-menu']
274 } else {
275 dockerModuleList = dockerModules()
276 }
277 customJobProperties = getCustomJobProperties()
278 }
279
280 xwikiDockerBuild {
281 configurations = dockerConfigurationList
282 modules = dockerModuleList
283 // Make sure that we don't reset the job properties!
284 jobProperties = customJobProperties
285 }
286 }
287 {{/code}}
288 )))
289
290 Pros/Cons of this approach (which is needed because Jenkins doesn't support more than one ##Jenkinsfile##) vs using a separate job:
291 * Pros:
292 ** No need to handle branches, done automatically by the ##Jenkinsfile## (creation and deletion). Also works for PRs.
293 * Cons:
294 ** Jenkins build history gets a bit messed up, because you're reusing the same job name but running possibly different builds. For example, test failure age will get reset every time a different type of build is ran, but at least individual test history is kept.
295
296 == Clover Pipeline ==
297
298 Used to generate the global test coverage for XWiki (over all XWiki Standard repositories). See [[Test Coverage>>Community.Testing.TestCoverage.WebHome]].
299
300 Example usage:
301
302 {{code language="groovy"}}
303 import org.xwiki.jenkins.Clover
304 node('docker') {
305 new Clover().generateGlobalCoverage([
306 [baseline: "20171222-1835", fail: false],
307 [baseline: "20190101-2330", fail: true]
308 ])
309 }
310 {{/code}}
311
312 = Maintainer's guide =
313
314 == Prerequisites ==
315
316 * You need to have an account on [[maven.xwiki.org>>http://maven.xwiki.org/]] (this is the machine hosting XWiki's remote repository) and you'll need a key setup on your account there so that you can ssh to it without having to enter username or password.
317 * You need to have an administrator account on [[ci.xwiki.org>>http://ci.xwiki.org/signup]].
318
319 == Functional tests ==
320
321 === Debugging ===
322
323 **Start remote Firefox**
324
325 * For example, to start firefox on machine ##ks4##: {{code}}ssh -X <username>@<ks4 hostname>{{/code}}
326 ** For this to work you need to ensure that ##X11Forwarding## is set to ##yes## in ##/etc/ssh/sshd_config## on the remote machine:(((
327 {{code}}
328 sudo vi /etc/ssh/sshd_config
329 {{/code}}
330
331 Then reload sshd: {{code}}sudo /etc/init.d/ssh reload{{/code}}
332 )))
333
334 **Connect to the XVNC server**
335
336 * Establish a SSH tunnel between your computer and the server on port 5901 (//ssh -L 5901:localhost:5901 <[email protected] address>//)
337 * Use your favorite VNC client to connect to the XVNC server
338 ** Host : localhost
339 ** Display : 1
340 ** Password : same password as for XWiki SAS wifi access points
341 * See the functional tests live.
342
343 * Work in progress to try to connect to an agent not exposing an IP address externally (by going through maven.xwiki.org):
344 ** {{code}}ssh -L 5901:192.168.1.119:5901 -o ProxyCommand="ssh [email protected] nc -w1 %h %p" [email protected] -i ~/.ssh/id_rsa_mavenxwikiorg{{/code}}
345 ** Where ##~/.ssh/id_rsa_mavenxwikiorg## is the private key on maven.xwiki.org, used to connect to 192.168.1.119 (ip of agent 2-4 in this example)
346 ** Not working yet since I can't connect with my local vnc client.

Get Connected