Wiki source code of XWiki's Continuous Build
Version 18.2 by Vincent Massol on 2019/03/03 15:49
Show last authors
| author | version | line-number | content |
|---|---|---|---|
| 1 | {{box cssClass="floatinginfobox" title="**Contents**"}} | ||
| 2 | {{toc/}} | ||
| 3 | {{/box}} | ||
| 4 | |||
| 5 | = General = | ||
| 6 | |||
| 7 | XWiki has a [[Continuous Integration>>http://www.martinfowler.com/articles/continuousIntegration.html]] setup to ensure XWiki's code is built at all times (i.e at every code check in). This also allows developers to only build the module they want and they'll have the other XWiki module dependencies downloaded from the [[XWiki remote repository>>http://maven.xwiki.org]]. | ||
| 8 | |||
| 9 | {{image reference="xwikibuildprocess.png"/}} | ||
| 10 | |||
| 11 | We use the following tools: | ||
| 12 | |||
| 13 | * [[GitHub>>https://github.com/xwiki]] is used as our [[SCM>>http://en.wikipedia.org/wiki/Source_Code_Management]] | ||
| 14 | * [[Maven>>http://maven.apache.org/]] is used for [[building XWiki>>Community.Building]] | ||
| 15 | * [[Jenkins>>http://jenkins-ci.org/]] is used as our Continuous Integration (CI) tool (set up at [[http://ci.xwiki.org/]]) | ||
| 16 | |||
| 17 | We use a technique called [[binary dependency build>>http://web.archive.org/web/20090423073100/http://blogs.codehaus.org/people/vmassol/archives/000953_binary_dependency_builds.html]] which allows to check out only the module a developer wishes to work on and he'll automatically get the latest fresh binary dependencies built by our CI tool. | ||
| 18 | |||
| 19 | = Jenkins Pipelines = | ||
| 20 | |||
| 21 | == Main Pipeline == | ||
| 22 | |||
| 23 | We have developed a {{scm project="xwiki-jenkins-pipeline" path="vars/xwikiModule.groovy"}}Jenkins Pipeline script{{/scm}} to build XWiki projects (can be used for core and contrib projects). It has the following features: | ||
| 24 | |||
| 25 | * Automatically build branches when new branches are added (or Pull Requests). Automatically stop building branches when they are removed from the SCM. | ||
| 26 | * Automatically use the right version of Java (and proper memory configuration) | ||
| 27 | * Use Jenkins's Xvnc plugin to have a Display for functional tests | ||
| 28 | * Use the "deploy" maven goal for "master" and "stable-*" branches only and "install" for the rest | ||
| 29 | * [[Attach the screenshot>>http://massol.myxwiki.org/xwiki/bin/view/Blog/AttachFailingTestPipeline]] of a failing XWiki Selenium test to the failed test's description | ||
| 30 | * Check for false positives for known cases of failures not related to code + [[check for test flickers>>http://massol.myxwiki.org/xwiki/bin/view/Blog/FlakyTestTool]] | ||
| 31 | * Send mails for build failures | ||
| 32 | |||
| 33 | To use is for a project, add a ##Jenkinsfile## with the following minimal configuration: | ||
| 34 | |||
| 35 | {{code language="groovy"}} | ||
| 36 | /* | ||
| 37 | * See the NOTICE file distributed with this work for additional | ||
| 38 | * information regarding copyright ownership. | ||
| 39 | * | ||
| 40 | * This is free software; you can redistribute it and/or modify it | ||
| 41 | * under the terms of the GNU Lesser General Public License as | ||
| 42 | * published by the Free Software Foundation; either version 2.1 of | ||
| 43 | * the License, or (at your option) any later version. | ||
| 44 | * | ||
| 45 | * This software is distributed in the hope that it will be useful, | ||
| 46 | * but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| 47 | * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU | ||
| 48 | * Lesser General Public License for more details. | ||
| 49 | * | ||
| 50 | * You should have received a copy of the GNU Lesser General Public | ||
| 51 | * License along with this software; if not, write to the Free | ||
| 52 | * Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA | ||
| 53 | * 02110-1301 USA, or see the FSF site: http://www.fsf.org. | ||
| 54 | */ | ||
| 55 | |||
| 56 | // It's assumed that Jenkins has been configured to implicitly load the vars/xwikiModule.groovy library which exposes | ||
| 57 | // the "xwikiModule" global function/DSL. | ||
| 58 | // Note that the version used is the one defined in Jenkins but it can be overridden as follows: | ||
| 59 | // @Library("XWiki@<branch, tag, sha1>") _ | ||
| 60 | // See https://github.com/jenkinsci/workflow-cps-global-lib-plugin for details. | ||
| 61 | |||
| 62 | xwikiModule { | ||
| 63 | } | ||
| 64 | {{/code}} | ||
| 65 | |||
| 66 | For more elaborate configuration, see: | ||
| 67 | * {{scm project="xwiki-commons" path="Jenkinsfile"}}xwiki-commons's Jenkinsfile{{/scm}} | ||
| 68 | * {{scm project="xwiki-platform" path="Jenkinsfile"}}xwiki-platform's Jenkinsfile{{/scm}} | ||
| 69 | |||
| 70 | = Maintainer's guide = | ||
| 71 | |||
| 72 | == Prerequisites == | ||
| 73 | |||
| 74 | * You need to have an account on [[maven.xwiki.org>>http://maven.xwiki.org/]] (this is the machine hosting XWiki's remote repository) and you'll need a key setup on your account there so that you can ssh to it without having to enter username or password. | ||
| 75 | * You need to have an administrator account on [[ci.xwiki.org>>http://ci.xwiki.org/signup]]. | ||
| 76 | |||
| 77 | == Functional tests == | ||
| 78 | |||
| 79 | === General === | ||
| 80 | |||
| 81 | * If your project includes [[functional tests>>http://selenium.openqa.org/]] don't check "Run Xvnc during build" in the project configuration, we have a XVNC server always running on the machine and jenkins is aware of it (DISPLAY :1.0). | ||
| 82 | |||
| 83 | === Debugging === | ||
| 84 | |||
| 85 | **Connect to the XVNC server** | ||
| 86 | |||
| 87 | * Establish a SSH tunnel between your computer and the server on port 5901 (//ssh -L 5901:localhost:5901 [email protected]//) | ||
| 88 | * Use your favorite VNC client to connect to the XVNC server | ||
| 89 | ** Host : localhost | ||
| 90 | ** Display : 1 | ||
| 91 | ** Password : same password as for XWiki SAS wifi access points | ||
| 92 | * See the functional tests live. | ||
| 93 | |||
| 94 | === Jenkins jamming up === | ||
| 95 | |||
| 96 | Occasionally (especially under high load) Jenkins will have an SCM polling thread jam up due to [[JENKINS-5413>>https://issues.jenkins-ci.org/browse/JENKINS-5413]]. This will prevent any further builds but it can be remedied by running the following script on the master's script console. | ||
| 97 | |||
| 98 | {{code language="groovy"}} | ||
| 99 | int i = 0; | ||
| 100 | Thread.getAllStackTraces().keySet().each(){ item -> | ||
| 101 | if( item.getName().contains("SCM polling") && | ||
| 102 | item.getName().contains("waiting for hudson.remoting")){ | ||
| 103 | println("THREAD: " + item); | ||
| 104 | item.interrupt(); | ||
| 105 | i++ | ||
| 106 | } | ||
| 107 | } | ||
| 108 | println("count = " + i); | ||
| 109 | {{/code}} |
