Show last authors
1 {{box cssClass="floatinginfobox" title="**Contents**"}}
2 {{toc/}}
3 {{/box}}
4
5 {{error}}
6 * Use Java 8 (Java 9+ is not supported yet)
7 * Make sure you install a Maven version >= 3.5.0. Note that our [[CI>>http://ci.xwiki.org]] builds XWiki using Maven 3.6.1 and thus this is our recommended version.
8 * Windows users have several choices for getting past the [[long paths issue>>https://github.com/msysgit/git/pull/122]] (255 char limitation on Windows) and building XWiki:
9 ** Use either [[GitHub Desktop>>https://desktop.github.com/]] or [[Git for Windows>>https://git-for-windows.github.io/]] and set ##core.longpaths## to ##True##: {{code}}git config --global core.longpaths true{{/code}}.
10 ** Use [[Cygwin>>https://www.cygwin.com/]] which works out of the box.
11 * If you're using openJDK, you must use a [[version equal or newer than 8u191-b12>>https://stackoverflow.com/questions/53010200/maven-surefire-could-not-find-forkedbooter-class/53016532#53016532]].
12 {{/error}}
13
14 = Understanding the directory structure =
15
16 Before you start it's good to have some minimal understanding of [[how Maven works>>http://maven.apache.org/users/index.html]]. You probably won't need it if everything goes fine but you'll need that knowledge if something breaks! ;-)
17
18 The first thing to understand is the [[directory structure we use>>SourceRepository]].
19
20 = Checking out the sources =
21
22 Use your favorite SCM client to [[check out the Project>>SourceRepository]] you wish to build.
23
24 = Installing Maven =
25
26 * Install [[Maven 3.5.4 or greater>>http://maven.apache.org/]]. Create an ##M2_HOME## environment variable pointing to where you have installed Maven. Add ##M2_HOME/bin## to your PATH environment variable.(((
27 {{info}}
28 For more information on how to install Maven or how to use it check this [[excellent Maven book>>http://www.sonatype.com/books/mvnref-book/reference/]].
29 {{/info}}
30 )))
31 * Make sure you give Maven's JVM enough memory. XWiki's build uses Aspects and GWT compilers which need lots of memory. A good value is to give the JVM 1024MB. To do that set an environment variable named ##MAVEN_OPTS##. For example on Unix add the following to your shell startup script (depending on your shell you might need to prefix the variable with ##export##):(((
32 {{code language="none"}}
33 MAVEN_OPTS="-Xmx1024m"
34 {{/code}}
35
36 Note that XWiki's tests are executed in a separate JVM which requires 1024MB of memory. Thus, in total, you need to have 2GB of free RAM to build XWiki.
37 )))
38 * Create a ##~~/.m2/settings.xml## file with the XWiki custom remote repository defined as shown below (if you're using a Repository Manager such as Nexus or Artifactory, you should add the XWiki Maven Remote repositories in there instead). If you are on Windows, create the directory ##.m2## in your home directory, e.g. ##C:\Documents and Settings\Administrator\.m2##. If you cannot do this from Windows Explorer, open a command line and use ##md .m2##.(((
39 {{info}}
40 You might be wondering why we don't push all XWiki artifacts to Maven Central and thus remove the need to declare the XWiki Maven Repositories.
41
42 Actually we do this for XWiki Commons and XWiki Rendering. Since these 2 projects contain libraries that are independent of the concept of wikis, this makes it very easy for everyone to use them in their own projects.
43
44 We haven't done so for XWiki Platform and XWiki Enterprise this far because Maven Central imposes a requirement that we cannot fulfil at the moment: all artifacts deployed there must not have dependencies on other artifacts that are not present on Maven Central. And we do have a lot of such 3rd party dependencies. Some of them do not even exist in any Maven Repository and we put them in our [[XWiki Externals Repository>>http://maven.xwiki.org/externals/]].
45
46 Note that we recommend using a Maven Repository Manager to make it easier to maintain your Maven Repositories setup.
47 {{/info}}
48
49 {{code language="xml"}}
50 <settings>
51 <profiles>
52 <profile>
53 <id>xwiki</id>
54 <repositories>
55 <repository>
56 <id>xwiki-snapshots</id>
57 <name>XWiki Nexus Snapshot Repository Proxy</name>
58 <url>http://nexus.xwiki.org/nexus/content/groups/public-snapshots</url>
59 <releases>
60 <enabled>false</enabled>
61 </releases>
62 <snapshots>
63 <enabled>true</enabled>
64 </snapshots>
65 </repository>
66 <repository>
67 <id>xwiki-releases</id>
68 <name>XWiki Nexus Releases Repository Proxy</name>
69 <url>http://nexus.xwiki.org/nexus/content/groups/public</url>
70 <releases>
71 <enabled>true</enabled>
72 </releases>
73 <snapshots>
74 <enabled>false</enabled>
75 </snapshots>
76 </repository>
77 </repositories>
78 <pluginRepositories>
79 <pluginRepository>
80 <id>xwiki-snapshots</id>
81 <name>XWiki Nexus Plugin Snapshot Repository Proxy</name>
82 <url>http://nexus.xwiki.org/nexus/content/groups/public-snapshots</url>
83 <releases>
84 <enabled>false</enabled>
85 </releases>
86 <snapshots>
87 <enabled>true</enabled>
88 </snapshots>
89 </pluginRepository>
90 <pluginRepository>
91 <id>xwiki-releases</id>
92 <name>XWiki Nexus Plugin Releases Repository Proxy</name>
93 <url>http://nexus.xwiki.org/nexus/content/groups/public</url>
94 <releases>
95 <enabled>true</enabled>
96 </releases>
97 <snapshots>
98 <enabled>false</enabled>
99 </snapshots>
100 </pluginRepository>
101 </pluginRepositories>
102 </profile>
103 </profiles>
104 <activeProfiles>
105 <activeProfile>xwiki</activeProfile>
106 </activeProfiles>
107 </settings>
108 {{/code}}
109
110 {{info}}
111 Depending on your geographical location, you should use closer Maven repositories to speed up your build process. For more information refer to the [[Maven Mirror Settings Guide>>http://maven.apache.org/guides/mini/guide-mirror-settings.html]].
112 {{/info}}
113 )))
114
115 = Building with Maven =
116
117 Now that you have installed Maven and checked out the [[Projects>>SourceRepository#HTopLevelProjects]] you wanted to work on, you'll want to build them or parts of them. A project is made of modules. To build a module in Maven just issue the command:
118
119 {{code}}
120 mvn clean install
121 {{/code}}
122
123 Thanks to XWiki's [[Continuous Integration setup>>DevelopmentPractices#HContinuousIntegration]] you can check out and build any single module (and its children) you wish without having to rebuild the other modules from source. Maven will pick the latest version of your module's dependencies from [[XWiki's Maven remote repositories>>http://maven.xwiki.org/]]. Of course if you have uncommitted changes in a dependent module, you'll want to build that module before. The build result is placed in two places:
124
125 * Your [[local maven repository>>http://maven.apache.org/guides/introduction/introduction-to-repositories.html]], for making the module available to other projects or modules using maven (even other XWiki modules take the needed libraries from this repository, and not directly from a "neighbor" directory)
126 * In a subdirectory of that module, called ##target##.
127
128 {{info}}
129 Some of the modules built by XWiki are very large (in the 40MB+ size) and sometimes you don't absolutely need the latest version to get downloaded as it'll take too long. In that case here's a tip: run Maven in 'no-snapshot-update' mode with ##mvn install -nsu##.
130 {{/info}}
131
132 {{warning}}
133 You might encounter a ##java.lang.OutOfMemoryError## error during ##aspectj:compile##. You must increase the maximum memory used by maven, by setting ##MAVEN_OPTS=-Xmx1024m## (or a larger amount of memory if needed).
134 {{/warning}}
135
136 == Using Profiles ==
137
138 Project builds define several [[Maven Profiles>>http://maven.apache.org/guides/introduction/introduction-to-profiles.html]]. We've standardized their names across Projects.
139
140 Here are the most common ones:
141
142 * ##legacy##: Includes the [[legacy modules>>DevelopmentPractices#HDeprecationRules]] in the build.
143 * ##integration-tests##: Executes integration and functional tests in the build.
144 ** ##docker##: Executes integration tests that are Docker-based. You'll need to have Docker installed for this to work.
145 * ##jetty##, ##glassfish##: Run the build for the specified container (##jetty## is the default when not specified).
146 * ##hsqldb## (Hypersonic SQL), ##mysql## (MySQL), ##pgsql## (PostgreSQL), ##derby## (Derby): Run the build for the specified database (##hsqldb## is the default when not specified).(((
147 {{warning}}
148 Hypersonic and Derby are embedded database so you won't need anything setup to execute the build for them. However for MySQL and PostgreSQL you'll need to have the database installed, set up and running before you can run the build with their profiles.
149 {{/warning}}
150 )))
151 * ##firefox## (Firefox), ##iexplore## (Internet Explorer), ##chrome## (Google Chrome), ##otherbrowser## (need to define ##browserPath## property to use that one): Run the functional tests on the specified browser (##firefox## is the default when not specified).
152 * ##dev##: only build wysiwyg for English/Firefox. Very useful to speed up the build of XWiki Platform
153 * ##distribution##: {{info}}Since 9.5RC1{{/info}} Also build distribution packages (the WAR, the flavor, the jetty/hsqldb package, etc)
154 ** ##flavor-integration-tests##: {{info}}Since 9.5RC1{{/info}} Execute integration and functional tests related to the the default flavor located in ##xwiki-platform##.
155
156 More "exotic" ones:
157
158 * ##release##: Used by the Maven Release plugin when releasing XWiki. It generates Javadoc and Source artifacts. It also checks that the Java version used is JDK6, that the Javadoc exists and it signs artifacts using GPG (this is a good practice and a [[requirement for being able to upload some of our artifacts to the Maven Central Repository>>http://maven.apache.org/guides/mini/guide-central-repository-upload.html]]).
159 * ##snapshot##: {{info}}Since 11.8RC1{{/info}} To be used when building snapshots (so never on a release!). Enabling this profile does the following:
160 ** The resulting XWiki Standard packagings includes the XWiki snapshots Extension Repository Source preconfigured and anyone downloading it for local testing will not have to enable it by hand. Distribution Wizard and Extension Manager will thus install the latest published snapshot extension builds and not fail completely saying that it can not find snapshot versions. Note that before version 11.8RC1, the profile used to be named ##snapshotModules## and has been renamed to be more generic.
161 ** Defines automatically the XWiki Maven Repository so that you don't have to modify your Maven ##settings.xml## to have it there. This is useful for tools such as LGTM which need to have the XWiki Maven Remote repo configured. It'll also be useful for simpler developer onboarding when we enable this profile automatically when the version in the ##pom.xml## ends with ##SNAPSHOT## (when we find how to do it! ;)).
162 * ##m2e##: For [[m2eclipse>>http://www.eclipse.org/m2e/]] users. It sets the Eclipse output directory to ##target-eclipse## (instead of ##target##) to prevent race conditions between Maven within Eclipse and Maven on the command line.
163 * ##unix##, ##mac##, ##windows##: These profiles are automatically activated depending on the OS the build is running on. These profiles are useful for the Installers and for functional tests to decide how to start XWiki.
164 * ##macprofiler##, ##winprofiler##: Start [[XWiki for Profiling>>Profiling]] (they require that you set a ##profilePath## property in your ##settings.xml## or on the command line)
165 * ##gwt-test-manual##: allow running GWT unit tests manually
166 * ##debug##: remove JS minification.
167 * ##standalone##: Builds the Rendering Standalone artifact. Note that this profile is activated automatically when performing a release.
168 * ##quality##: Runs quality checks that take a long time to execute (e.g. checks Jacoco TPC thresholds)
169 * ##clover##: A profile to use when [[generating Test Coverage reports>>Testing#HTestCoverage]] with [[Clover>>http://www.atlassian.com/software/clover/overview]]. This profile skips execution of Checkstyle, Backward Compatibility and Enforcer plugin checks to speed up the coverage generation and because of potential conflicts between tools. Note that even though we've started using Jacoco for our ##quality## profile we still support Clover. The Clover reports are much nicer than the Jacoco ones and it's thus still useful to be able to generate them.
170
171 For example, if you use to include legacy modules and run all integration and functional tests and use Jetty/HSQLDB, you would use:
172
173 {{code}}
174 mvn clean install -Plegacy,integration-tests,hsqldb,jetty
175 {{/code}}
176
177 == Relationship between XWiki Enterprise and XWiki Platform ==
178
179 XWiki Platform is a generic platform and API for building flexible extensible wikis and web applications. XWiki Enterprise is an implementation of the platform geared toward making a wiki site for teams to share information and synchronize their efforts although it can be and has been used for a wide variety of different use cases like this website.
180
181 What does this mean to the developer?
182 If you wish to make changes in the core of XWiki, you are probably going to want to work on XWiki Platform. You can download and compile XWiki Enterprise and while compiling, you will notice Maven downloads a big 80MB+ file, this is the platform. Maven is smart enough to know that you don't have a copy of the platform and will download it from the XWiki Maven repository. Once it has been downloaded the first time, it is stored on your hard drive and you won't have to download it again //until// changes are made in the platform.
183
184 If you want to work on the core or if you have limited bandwidth, then you are well advised to get the platform sources. Then you can use the SCM's update command to get the latest copy without downloading the entire platform all over again. If you download the platform and compile it then you can compile XWiki Enterprise, Maven will use your newly compiled copy of the platform rather than downloading it, Maven is smart.
185
186 = Building with Docker =
187
188 XWiki offers a [[Docker image to build XWiki>>https://github.com/xwiki/xwiki-docker-build/tree/master/build]], that is used by [[XWiki's CI>>https://ci.xwiki.org]]. It can also be used to build XWiki locally very easily. If you want to use it, you only need to install Docker and you don't need to have Maven installed or any of the other needed tools need to run XWiki's build (like a given version of Firefox for some old functional tests).
189
190 Check how to [[build with the XWiki Docker build image>>https://github.com/xwiki/xwiki-docker-build/tree/master/build]].
191
192 = Automatic Checks =
193
194 The XWiki build performs some automated checks:
195
196 * Style checks with Checkstyle. To skip: ##-Dxwiki.checkstyle.skip=true##. We also have custom Checkstyle checks:
197 ** Verify that ##@Unstable## annotation don't stay too long (and that a ##@since## annotation is present too)
198 ** Verify that Script Services are not located in the ##internal## package
199 ** Verify that ##@since## javadoc tags have the correct format (and that they don't contain several versions per tag). Example of correct format: ##@since 10.8RC1##.
200 * Verify header licenses
201 * Backward compatibility checks with [[revapi>>http://revapi.org/]]. To skip: ##-Dxwiki.revapi.skip=true##.
202 * Maven Enforcer checks (To skip: ##-Dxwiki.enforcer.skip=true##):
203 ** Verify that all plugins have versions specified
204 ** Verify that we don't use Commons Lang < 3 (i.e. that ##commons-lang:commons-lang## artifact is forbidden)
205 ** Verify the correct JUnit artifact is used (##junit-dep## and not ##junit##)
206 ** Verify we don't use Commons Logging or Log4j (since we use SLF4J)
207 ** Verify that the Java version used to release XWiki is correct (e.g. Java 8 starting with XWiki 8.1, Java 7 for XWiki 6.0 and Java 6 before)
208 ** Verify that Javadoc exist (in the release profile, this is a Maven Central requirement)
209 ** In Commons: Verify that Commons modules don't have a dependency on Rendering and Platform modules
210 ** In Rendering: Verify that Rendering modules don't have a dependency on Platform modules
211 * Spoon checks (To skip: ##-Dxwiki.spoon.skip=true##):
212 ** Verify none of the listed methods are called in our Java code
213 ** Verify that ##components.txt## contains all Components
214 ** Verify that JUnit4 and JUnit5 APIs are not both used at the same time in a test
215 ** Verify that the @Inject annotation is done on the component role (fails if not done by error on the component implementation, unless it's using the ##roles## attribute of the ##@Component## annotation and it points to itsself). {{info}}Since 12.4RC1{{/info}}
216 * For XARs, verify format and content using the [[XAR Plugin>>XARPlugin]]
217 * Verify that Test Percentage Coverage don't go down
218 ** This is achieved using Jacoco and the check is done at each module level using a Maven property named ##xwiki.jacoco.instructionRatio##.
219 ** Note: We also perform a global coverage check but [[it's done only in the CI>>dev:Community.ContinuousBuild#HCloverPipeline]]
220 * We check automatically that we don't have wiki pages that require Programming Rights unduly. This is exercised during functional tests. For more details see [[Testing documentation>>Community.Testing||anchor="HFunctionalTesting"]].
221 * Verify Test output
222 ** Verify that JUnit tests don't output content to stdout or stderr. Tests are supposed to either configure the code to not output anything or if it's expected then they're supposed to [[catch the logs and assert them>>dev:Community.Testing.JavaUnitTesting.WebHome#HCapturingandAssertingLogs]]. Note that this check is also executing for JUnit5 tests executed in your IDE (if the ##pom.xml## file in the current directory doesn't have the ##xwiki.surefire.captureconsole.skip## Maven property set to true).
223 ** Verify that functional tests don't contain errors or warnings when they execute. See [[Docker Tests>>Community.Testing.DockerTesting.WebHome||anchor="HStdout2Fstderrvalidationerrors"]] (for JUnit5) and [[Selenium2 Tests>>Community.Testing.WebHome||anchor="HBestpractices"]] (for JUnit4).
224
225 = Tips =
226
227 == Skipping Tests ==
228
229 While you should almost never do that, here's a tip for the rare occasions when you'll need to build something fast:
230
231 {{code}}
232 mvn install -Dtest=none -DfailIfNoTests=false
233 {{/code}}
234
235 Alternatives are ##mvn install -DskipTests=true## or ##mvn install -Dmaven.test.skip=true##. Refere to the Maven documentation to understand the differences.
236
237 = Troubleshooting =
238
239 == Dealing with Out-of-Memory Errors ==
240
241 If you get an ##OutOfMemoryError## then increase the maximum heap space by setting the ##MAVEN_OPTS## environment variable. For example:
242
243 {{code language="none"}}
244 MAVEN_OPTS="-Xmx1024m"
245 {{/code}}
246
247 == Error in Windows installer build: Windres error ==
248
249 Launch4J is using a binary called Windres which requires that the environment is NOT set to use UTF-8. If you get the following error check your LANG environment variable:
250
251 {{code language="none"}}
252 Embedded error: net.sf.launch4j.BuilderException: net.sf.launch4j.ExecException: Exec failed(1): /var/tmp/installers/windows/target/dependency/bin/windres --preprocessor=cat -J rc -O coff -F pe-i386 /tmp/launch4j10018rc /tmp/launch4j10019o
253 {{/code}}
254
255 == Building behind a proxy ==
256
257 If you are connecting to the Internet through a proxy then you need to modify your ##settings.xml## file so that Maven knows it:
258
259 {{code language="xml"}}
260 <settings>
261 <proxies>
262 <proxy>
263 <active>true</active>
264 <protocol>http</protocol>
265 <host>host</host>
266 <port>port</port>
267 <username>uname</username>
268 <password>password</password>
269 </proxy>
270 </proxies>
271 <!--other tags-->
272 </settings>
273 {{/code}}
274
275 == Errors about missing resources ==
276
277 Some people have reported problems when building XWiki for the first time, or the first time after a version update. The error message is loosely looking like:
278
279 {{code}}
280 [...]
281 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.4:process (xwiki-license-resources) on project [[some-xwiki-project-here]]: Resources archive cannot be found. Could not find artifact org.xwiki.commons:[[some-xwiki-commons-lib]]:jar:x.y.z-SNAPSHOT
282 [ERROR]
283 [ERROR] Try downloading the file manually from the project website.
284 [ERROR]
285 [ERROR] Then, install it using the command:
286 [ERROR] mvn install:install-file -DgroupId=org.xwiki.commons -DartifactId=[[some-xwiki-commons-lib]] -Dversion=x.y.z-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file
287 [ERROR]
288 [ERROR] Alternatively, if you host your own repository you can deploy the file there:
289 [ERROR] mvn deploy:deploy-file -DgroupId=org.xwiki.commons -DartifactId=[[some-xwiki-commons-lib]] -Dversion=x.y.z-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
290 [...]
291 {{/code}}
292
293 This is probably a problem with the maven version; try updating to the recommended maven version and retry.
294 \\Alternatively just call the maven build a second (or possible even a third time); the problem usually goes away for consecutive builds.
295
296 If the problem does not go away, check if http://nexus.xwiki.org/ is running. If it is not, please wait a moment until it comes back. (As this is a rather critical part of the infrastructure, you can be pretty sure it gets fixed quickly.)
297
298 = Build Tools =
299
300 We have developed some build tools in the form of Maven plugins. They are located in the [[##commons/tools##>>https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-tools]] and [[##platform/xwiki-tools/##>>https://github.com/xwiki/xwiki-platform/tree/master/xwiki-platform-tools]] directories. Here are descriptions of some of them:
301
302 * ##xwiki-packager-plugin##: Takes XWiki pages defined as XML files and located in ##src/main/resources/## and load them into a database (defined by ##src/main/packager/hibernate.cfg.xml##). An example usage is available [[here>>https://github.com/xwiki/xwiki-platform/tree/master/xwiki-platform-tools/xwiki-platform-tool-packager-plugin/src/it/basic-import-export]].
303 * ##[[xwiki-xar-plugin>>XARPlugin]]##: Takes XWiki pages defined as XML files and located in ##src/main/resources/## and generates a XAR (ZIP format) out of them.
304 * ##[[xwiki-enterprise-archetype>>XEArchetype]]##: A Maven Archetype to create a Maven project extending XWiki Enterprise. This is useful is you're planning to extend XWiki Enterprise and create a distribution for your product.
305 * ##[[xwiki-extension-plugin>>ExtensionPlugin]]##: Various extension oriented mojos for Maven build. Current mostly about generating extensions descriptors (.xed) in various situations (WAR dependencies, etc.).
306 * ##[[xwiki-commons-tool-webjar-handlers>>WebJarHandler]]##: A handler for Maven type ##webjar## to use to produce explicitly typed and standard webjar artifacts.
307 * ##[[xwiki-platform-tool-provision-plugin>>ProvisionPlugin]]##: Installs Extensions into a running XWiki instance
308 * ##[[xwiki-commons-tool-remote-resource-plugin>>XWikiRemoteResourcePlugin]]##: A more memory friendly alternative to standard Maven Remote Resource Plugin
309
310 Also, you can find some contrib tools:
311
312 * [[Off-line XWiki Repository Packager Maven Plugin>>https://github.com/xwiki-contrib/maven-plugin-offline-xwiki-repository-packager]]
313
314 = Building in Eclipse =
315
316 * [[BuildingInEclipse]]
317
318 = Building in IntelliJ IDEA =
319
320 * [[BuildingInIdea]]
321
322 = Creating a Build for your own Modules =
323
324 == Generating a XAR ==
325
326 Follow these steps:
327
328 * Put the XWiki XML files into the ##src/main/resources## directory
329 * Use the ##xar## packaging in the your ##pom.xml##.
330 * Make sure the XAR packaging extension is registered in Maven.
331
332 For example:
333
334 {{code language="xml"}}
335 <project...>
336 ...
337 <packaging>xar</packaging>
338 ...
339 <build>
340 <extensions>
341 <!-- Needed to add support for the "xar" packaging -->
342 <extension>
343 <groupId>org.xwiki.commons</groupId>
344 <artifactId>xwiki-commons-tool-xar-handlers</artifactId>
345 <version>(version to use)</version>
346 </extension>
347 </extensions>
348 </build>
349 ...
350 </project>
351 {{/code}}
352
353 Make sure you run ##mvn xar:format## to ensure your XWiki XML files are properly formatted, following [[XWiki XML files best practices>>XWikiXMLFilesCodeStyle]].
354
355 = Building a patched version =
356
357 If you want to patch a part of XWiki without compiling the entire software, you may take a look at the following [[blog article>>http://hole.tuziwo.info/patch-a-dependency-of-maven.html]]. It explains how to patch and compile only the JAR archive you want, without building all of the JAR archives of XWiki.

Get Connected