XWiki's Continuous Build

Last modified by Vincent Massol on 2017/11/08 13:19

General

XWiki has a Continuous Integration setup to ensure XWiki's code is built at all times (i.e at every code check in). This also allows developers to only build the module they want and they'll have the other XWiki module dependencies downloaded from the XWiki remote repository.

xwikibuildprocess.png

We use the following tools:

We use a technique called binary dependency build which allows to check out only the module a developer wishes to work on and he'll automatically get the latest fresh binary dependencies built by our CI tool.

Jenkins Pipeline

We have developed a Jenkins Pipeline script to build projects. It has the following features:

  • Automatically build branches when new branches are added (or Pull Requests). Automatically stop building branches when they are removed from the SCM.
  • Automatically use the right version of Java (and proper memory configuration)
  • Use Jenkins's Xvnc plugin to have a Display for functional tests
  • Use the "deploy" maven goal for "master" and "stable-*" branches only and "install" for the rest
  • Attach the screenshot of a failing XWiki Selenium test to the failed test's description
  • Check for false positives for known cases of failures not related to code + check for test flickers
  • Send mails for build failures

Note that currently this script is used by the XWiki Contrib Job (which is a job of type GitHub Organization).

Our goal is to use this script also for the XWiki Github organization in the future. At the moment we're using "manual" jobs.

Maintainer's guide

Prerequisites

  • You need to have an account on maven.xwiki.org (this is the machine hosting XWiki's remote repository) and you'll need a key setup on your account there so that you can ssh to it without having to enter username or password.
  • You need to have an administrator account on ci.xwiki.org.

Functional tests

General

  • If your project includes functional tests don't check "Run Xvnc during build" in the project configuration, we have a XVNC server always running on the machine and jenkins is aware of it (DISPLAY :1.0).

Debugging

Connect to the XVNC server

  • Establish a SSH tunnel between your computer and the server on port 5901 (ssh -L 5901:localhost:5901 [email protected])
  • Use your favorite VNC client to connect to the XVNC server
    • Host : localhost
    • Display : 1
    • Password : same password as for XWiki SAS wifi access points
  • See the functional tests live.

Jenkins jamming up

Occasionally (especially under high load) Jenkins will have an SCM polling thread jam up due to JENKINS-5413. This will prevent any further builds but it can be remedied by running the following script on the master's script console.

int i = 0;
Thread.getAllStackTraces().keySet().each(){ item ->
 if( item.getName().contains("SCM polling") &&
      item.getName().contains("waiting for hudson.remoting")){
     println("THREAD: " + item);
     item.interrupt();
     i++
 }
}
println("count = " + i);
Tags:
Created by Jean-Vincent Drean on 2008/10/14 18:54
   

Get Connected