Wiki source code of XWiki's Continuous Build

Version 16.4 by Vincent Massol on 2017/11/08 12:58
Show last authors
1 {{box cssClass="floatinginfobox" title="**Contents**"}}
2 {{toc/}}
3 {{/box}}
4
5 = General =
6
7 XWiki has a [[Continuous Integration>>http://www.martinfowler.com/articles/continuousIntegration.html]] setup to ensure XWiki's code is built at all times (i.e at every code check in). This also allows developers to only build the module they want and they'll have the other XWiki module dependencies downloaded from the [[XWiki remote repository>>http://maven.xwiki.org]].
8
9 {{image reference="xwikibuildprocess.png"/}}
10
11 We use the following tools:
12
13 * [[GitHub>>https://github.com/xwiki]] is used as our [[SCM>>http://en.wikipedia.org/wiki/Source_Code_Management]]
14 * [[Maven>>http://maven.apache.org/]] is used for [[building XWiki>>Community.Building]]
15 * [[Jenkins>>http://jenkins-ci.org/]] is used as our Continuous Integration (CI) tool (set up at [[http://ci.xwiki.org/]])
16
17 We use a technique called [[binary dependency build>>http://web.archive.org/web/20090423073100/http://blogs.codehaus.org/people/vmassol/archives/000953_binary_dependency_builds.html]] which allows to check out only the module a developer wishes to work on and he'll automatically get the latest fresh binary dependencies built by our CI tool.
18
19 = Jenkins Pipeline =
20
21 We have developed a {{scm project="xwiki-jenkins-pipeline" path="vars/xwikiModule.groovy"}}Jenkins Pipeline script{{/scm}} to build projects. It has the following features:
22
23 * Automatically build branches when new branches are added (or Pull Requests). Automatically stop building branches when they are removed from the SCM.
24 * Automatically use the right version of Java (and proper memory configuration)
25 * Use Jenkins's Xvnc plugin to have a Display for functional tests
26 * Use the "deploy" maven goal for "master" and "stable-*" branches only and "install" for the rest
27 * [[Attach the screenshot>>http://massol.myxwiki.org/xwiki/bin/view/Blog/AttachFailingTestPipeline]] of a failing XWiki Selenium test to the failed test's description
28 * Check for false positives for known cases of failures not related to code + [[check for test flickers>>http://massol.myxwiki.org/xwiki/bin/view/Blog/FlakyTestTool]]
29 * Send mails for build failures
30
31 Note that currently this script is used by the [[XWiki Contrib Job>>http://ci.xwiki.org/job/XWiki%20Contrib/]] (which is a [[job of type GitHub Organization>>http://massol.myxwiki.org/xwiki/bin/view/Blog/JenkinsGitHubOrganization]]).
32
33 Our goal is to use this script also for the XWiki Github organization in the future. At the moment we're using "manual" jobs.
34
35 = Maintainer's guide =
36
37 == Prerequisites ==
38
39 * You need to have an account on [[maven.xwiki.org>>http://maven.xwiki.org/]] (this is the machine hosting XWiki's remote repository) and you'll need a key setup on your account there so that you can ssh to it without having to enter username or password.
40 * You need to have an administrator account on [[ci.xwiki.org>>http://ci.xwiki.org/signup]].
41
42 == Functional tests ==
43
44 === General ===
45
46 * If your project includes [[functional tests>>http://selenium.openqa.org/]] don't check "Run Xvnc during build" in the project configuration, we have a XVNC server always running on the machine and jenkins is aware of it (DISPLAY :1.0).
47
48 === Debugging ===
49
50 **Connect to the XVNC server**
51
52 * Establish a SSH tunnel between your computer and the server on port 5901 (//ssh -L 5901:localhost:5901 [email protected]//)
53 * Use your favorite VNC client to connect to the XVNC server
54 ** Host : localhost
55 ** Display : 1
56 ** Password : same password as for XWiki SAS wifi access points
57 * See the functional tests live.
58
59 === Wysiwyg functional tests ===
60
61 GWT seems to have an unusual way to manage input events, our WYSIWYG editor needs its container window to have a _real_ focus (Windowing System level) to catch them (at least on Linux and OSX). Wysiwyg functional tests execute [[a small C program>>attach:[email protected]]], the compiled program must be available at ///home/maven/xwindowfocus// on the continuous build server.
62
63 **Source : main.c**
64
65 {{code language="c"}}
66 /*
67 * See the NOTICE file distributed with this work for additional
68 * information regarding copyright ownership.
69 *
70 * This is free software; you can redistribute it and/or modify it
71 * under the terms of the GNU Lesser General Public License as
72 * published by the Free Software Foundation; either version 2.1 of
73 * the License, or (at your option) any later version.
74 *
75 * This software is distributed in the hope that it will be useful,
76 * but WITHOUT ANY WARRANTY; without even the implied warranty of
77 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
78 * Lesser General Public License for more details.
79 *
80 * You should have received a copy of the GNU Lesser General Public
81 * License along with this software; if not, write to the Free
82 * Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
83 * 02110-1301 USA, or see the FSF site: http://www.fsf.org.
84 */
85
86 #include <sys/types.h>
87 #include <stdio.h>
88 #include <stdlib.h>
89 #include <string.h>
90 #include <stdarg.h>
91
92 #include <X11/X.h>
93 #include <X11/Xlib.h>
94 #include <X11/Xutil.h>
95
96 int XWindowFocus(Display *display, char *window_name)
97 {
98 Window root;
99 Window query_root, query_parent;
100 Window *children_windows;
101 char *win_name;
102 unsigned int children_nb;
103 int match, i;
104
105 root = DefaultRootWindow(display);
106 // Retrieve all windows
107 XQueryTree(display, root, &query_root, &query_parent, &children_windows, &children_nb);
108 // Loop over windows
109 for (i = 0, match = 1; i < children_nb; i++)
110 {
111 XFetchName(display, children_windows[i], &win_name);
112 // Match window name
113 if (!strcmp(win_name, window_name))
114 {
115 // Set X focus on the matching window
116 XSetInputFocus(display, children_windows[i], RevertToNone, CurrentTime);
117 printf("Focus set on window \"%s\"\n", win_name);
118 match = 0;
119 }
120 XFree(win_name);
121 }
122 XFree(children_windows);
123
124 return match;
125 }
126
127 int main(int ac, char **av)
128 {
129 Display *display = XOpenDisplay(NULL);
130 int ret;
131
132 // Check arguments
133 if (ac < 2)
134 {
135 fprintf(stderr, "Usage : xwindowfocus <window_name>\n");
136 exit(1);
137 }
138
139 // Verify DISPLAY
140 if (display == NULL)
141 {
142 fprintf(stderr, "Error : failed to open DISPLAY\n");
143 exit(1);
144 }
145
146 ret = XWindowFocus(display, av[1]);
147
148
149 if (ret == 1)
150 {
151 fprintf(stderr, "Error : failed to find window\n");
152 }
153
154 exit(ret);
155 }
156 {{/code}}
157
158 === Jenkins jamming up ===
159
160 Occasionally (especially under high load) Jenkins will have an SCM polling thread jam up due to [[JENKINS-5413>>https://issues.jenkins-ci.org/browse/JENKINS-5413]]. This will prevent any further builds but it can be remedied by running the following script on the master's script console.
161
162 {{code language="groovy"}}
163 int i = 0;
164 Thread.getAllStackTraces().keySet().each(){ item ->
165 if( item.getName().contains("SCM polling") &&
166 item.getName().contains("waiting for hudson.remoting")){
167 println("THREAD: " + item);
168 item.interrupt();
169 i++
170 }
171 }
172 println("count = " + i);
173 {{/code}}

About

About

Support

Platform

User Guide

Admin Guide

Developer Guide

Projects

XWiki

Extensions

Other

Contribute

Status

Practices

Under the Hood

Get Involved

Get Connected