Building XWiki from sources

Contents
Make sure you install Maven 2.2.0 or greater. It might build fine using earlier versions but we're not supporting them and all the XWiki committers and our CI server are running 2.2.0 and greater.

Understanding the directory structure

Before you start it's good to have some minimal understanding of how Maven2 works. You probably won't need it if everything goes fine but you'll need that knowledge if something breaks! ;-)

The first thing to understand is the directory structure used.

Checking out the sources

Use your favorite Subversion client to check out the sources.

You can check out the whole source tree or only a single module you wish to build. Maven2 is powerful enough in that it'll always try to download the required dependencies from the remote repositories you have defined, so that you don't have to build the whole project from sources.

Installing Maven

  • Install Maven 2.1.0-M1 or greater. Create an M2_HOME environment variable pointing to where you have installed Maven. Add M2_HOME/bin to your PATH environment variable.
    For more information on how to install Maven or how to use it check this excellent Maven book.
  • Make sure you give Maven's JVM enough memory. XWiki's build uses Aspects and GWT compilers which need lots of memory. A good value is to give the JVM 600MB. To do that set an environment variable named MAVEN_OPTS. For example on Unix add the following to your shell startup script:
    MAVEN_OPTS=-Xmx500m -XX:MaxPermSize=256m
  • Create a ~/.m2/settings.xml file with the XWiki custom remote repository defined as shown below. If you are on Windows, create the directory .m2 in your home directory, e.g. C:Documents and SettingsAdministrator.m2. If you cannot do this from Windows Explorer, open a commandline and use md .m2.
    <settings>
     <profiles>
       <profile>
         <id>xwiki</id>
         <repositories>
           <repository>
             <id>xwiki-externals</id>
             <name>XWiki Maven2 Remote Repository for Third Party Dependencies</name>
             <url>http://maven.xwiki.org/externals</url>
             <releases>
               <enabled>true</enabled>
             </releases>
             <snapshots>
               <enabled>true</enabled>
             </snapshots>
           </repository>
           <repository>
             <id>xwiki-releases</id>
             <name>XWiki Maven2 Remote Repository for Releases</name>
             <url>http://maven.xwiki.org/releases</url>
             <releases>
               <enabled>true</enabled>
             </releases>
             <snapshots>
               <enabled>false</enabled>
             </snapshots>
           </repository>
           <repository>
             <id>xwiki-snapshots</id>
             <name>XWiki Maven2 Remote Repository for Snapshots</name>
             <url>http://maven.xwiki.org/snapshots</url>
             <releases>
               <enabled>false</enabled>
             </releases>
             <snapshots>
               <enabled>true</enabled>
             </snapshots>
           </repository>
         </repositories>
         <pluginRepositories>
           <pluginRepository>
             <id>xwiki-plugins-externals</id>
             <name>XWiki Maven2 Plugin Remote Repository for Third Party Plugins</name>
             <url>http://maven.xwiki.org/externals</url>
             <releases>
               <enabled>true</enabled>
             </releases>
             <snapshots>
               <enabled>true</enabled>
             </snapshots>
           </pluginRepository>
           <pluginRepository>
             <id>xwiki-plugins-releases</id>
             <name>XWiki Maven2 Plugin Remote Repository for Releases</name>
             <url>http://maven.xwiki.org/releases</url>
             <releases>
               <enabled>true</enabled>
             </releases>
             <snapshots>
               <enabled>false</enabled>
             </snapshots>
           </pluginRepository>
           <pluginRepository>
             <id>xwiki-plugins-snapshots</id>
             <name>XWiki Maven2 Plugin Remote Repository for Snapshots</name>
             <url>http://maven.xwiki.org/snapshots</url>
             <releases>
               <enabled>false</enabled>
             </releases>
             <snapshots>
               <enabled>true</enabled>
             </snapshots>
           </pluginRepository>
         </pluginRepositories>
       </profile>
     </profiles>
     <activeProfiles>
       <activeProfile>xwiki</activeProfile>
     </activeProfiles>
    </settings>
    Depending on your geographical location, you should use closer maven repositories to speed up your build process. For more information refer Maven Mirror Settings Guide.

Building with Maven

Projects are directories found under the XWiki SVN root. For example: platform, enterprise, manager, etc.

To build them, check them out and run mvn install.

Each project is made of modules. Thanks to XWiki's Continuous Integration setup you can check out and build any single module (and its children) you wish without having to rebuild the other modules from source. Maven will pick the latest version of your module's dependencies from XWiki's Maven remote repositories. Of course if you have uncommitted changes in a dependent module, you'll want to build that module before. The build result is placed in two places:

  • the local maven repository, for making the module available to other projects or modules using maven (even other XWiki modules take the needed libraries from this repository, and not directly from a "neighbor" directory)
  • and in a subdirectory of that module, called target.
Now some of the modules built by XWiki are very large (in the 40MB+ size) and sometimes you don't absolutely need the latest version to get downloaded as it'll take too long. In that case here's a tip: run Maven in 'no-snapshot-update' mode with mvn install -nsu.
You might encounter an java.lang.OutOfMemoryError during aspectj:compile. You must increase the maximum memory used by maven, by setting MAVEN_OPTS=-Xmx512m (or a different amount of memory).

You'll find below some common build tasks.

Building for a specific database

By default XWiki's Maven build will use Hypersonic SQL as the default database. If you wish to execute the build using a different database, you'll need to specify the correct database profile (-P<profile name>). Right now we support 4 profiles:

  • hsqldb: Hypersonic SQL. This is the default
  • mysql: MySQL
  • pgsql: PostGreSQL
  • derby: Apache Derby

For example to build a module for MySQL type mvn install -Pmysql in that module's directory.

Hypersonic and Derby are embedded database so you won't need anything setup to execute the build for them. However for MySQL and PostGreSQL you'll need to have the database installed, set up and running before you can run the build with their profiles. 

A simple way to do this is to define the necessary properties within {HOME}/.m2/settings.xml as explained above when configuring the maven mirrors. Check the property names required by looking at the corresponding profiles section in the platform's top level pom.xml file

Building on Windows

To build on Windows, you should specify the windows profile (-Pwindows).

Building XWiki Enterprise

Go in enterprise/ and type mvn install. It'll build it with a default config for HSQL DB. After the build is finished you'll find the standalone zip in enterprise/distribution/jetty/hsqldb/target/.

If you wish to build all the modules required for XWiki Enterprise and not the ones required by other products you can go at the top level and type mvn install"
The functional tests are not executed by default as the take a long time to execute and are meant mostly to be executed by the Continuous Integration tool. However if you wish to run them you can execute mvn install -Pxe,ci
The installer builds are not executed by default (they are executed on the Continuous Integration server though). If you wish to run them you can execute mvn install -Pxe,ci or better, go in xwiki-product-enterprise/installers and run mvn install.

The Relationship between XWiki Enterprise and XWiki Platform

XWiki Platform is a generic platform and API for building flexible extensible wikis and web applications. XWiki Enterprise is an implementation of the platform geared toward making a wiki site for teams to share information and synchronize their efforts although it can be and has been used for a wide variety of different use cases like this website.

What does this mean to the developer?
If you wish to make changes in the core of XWiki, you are probably going to want to work on XWiki Platform. You can download and compile XWiki Enterprise and while compiling, you will notice Maven downloads a big ~80MB file, this is the platform. Maven is smart enough to know that you don't have a copy of the platform and will download it from the XWiki Maven repository. Once it has been downloaded the first time, it is stored on your harddrive and you won't have to download it again until changes are made in the platform.

If you want to work on the core or if you have limited bandwidth, then you are well advised to download the platform. Then you can use the svn up command to get the latest copy without downloading the entire platform all over again. If you download the platform and compile it then you can compile XWiki Enterprise, Maven will use your newly compiled copy of the platform rather than downloading it, Maven is smart.

Building XWiki Platform

The platform is built much the same way as XWiki Enterprise but it is a large project and compiling can take a long time (over 1/2 hour.)
One of the reasons building the platform is slow is because the WYSIWYG module builds a version for each supported browser in each language. You can make it only build for English/Firefox by specifying -Pdev in the build command.
mvn install -Pdev
You can also build the platform faster if you skip the unit tests by building with the command:
mvn install -Dtest=none -DfailIfNoTests=false
Caution: If you have just made a patch, you should probably run the tests since breaking a test is embarrassing.
For added safety (especially when you have made changes which might affect the ability of other code to find the libraries it needs) you can compile with:
mvn clean install
and everything will be recompiled.
Press enter, go get a cup of coffee, and wait.

Executing the Automated Functional Tests for XWiki Enterprise

Go in enterprise/distribution-test and type mvn install. By default they are executed in Firefox. To run them in a different browser, use a profile. For example to run them in IE, type mvn install -Piexplore.

See the Testing page for more details.

Executing the Standard Web WAR quickly in development mode

Update 2010-03-07: The part below is a bit oldish and is probably not working any more. It needs someone to look at it. Please check the alternative below.

Go in platform/web/standard and type mvn install -Pjettyrun. This will execute it on Jetty with a HSQLDB database located in platform/web/standard/database (we need to improve the build to locate it elsewhere). So if you want to have any content displayed you'll need to put a HSQLDB there or simply import a XAR.

If you wish to use another database like MySQL, run mvn install -Pjettyrun,mysql. See the section above on building XWiki for a specific database for more information.

Once the build has started Jetty, open your browser and point it to http://localhost:8080/xwiki.

To debug in this mode, run MAVEN_OPTS='-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n' mvn install -Pjettyrun and attach you IDE to this process as in Debugging.

Note that this is a quick way of starting XWiki in development mode. However the real test is to start the XWiki Enterprise standalone distribution to ensure that everything works. This is achieved by running mvn install in enterprise/.

Here are some alternative strategies used by XWiki committers:

Building XEclipse

You'll need Eclipse 3.3.x (or above) and an associated PDE (Plugin Development Environment) in order to build XEclipse from source.

This section is about building XEclipse with maven. For instructions on how to build XEclipse from right within the Eclipse IDE, you may refer to Building in Eclipse

Building XEclipse as an Eclipse plugin

Change directory to plugins/org.xwiki.eclipse and run mvn install -DeclipseInstall=$ECLIPSE_DIR

Builing XEclipse as a standalone application

You will need the pde-maven-plugin.
Once this maven plugin is installed, please refer to plugins/org.xwiki.eclipse.rcp/README for further information about building instructions.

Building the WYSIWYG editor

Go in platform/web/wysiwyg and type mvn clean install. This will build the editor for all major browsers and for all supported locales. As a consequence the build will take a while and will require a lot of resources (both memory and processor). If you want to build the editor for only one browser and one locale (specified in WysiwygDev.gwt.xml, the development module descriptor) you should use the dev profile (-Pdev). To update the editor of an XWiki Enterprise instance you have to update the resources/js/xwiki/wysiwyg/xwe directory and the WEB-INF/lib/xwiki-web-wysiwyg.jar. You can find both in the war generated by the WYSIWYG build. Make sure they are compatible with the version of xwiki-core and gwt-servlet jars from your XWiki Enterprise instance.

By default unit tests are run using HtmlUnit. If you want to run the unit tests manually in a real browser, you have to use the manual profile (-Pmanual). At some point during the build you will be asked to point your browser to a specified URL.

You can run or debug the WYSIWYG editor in development mode by using the start_wysiwyg_noserver.sh and start_wysiwyg_noserver_debug.sh scripts. They connect to an existing XWiki Enterprise instance. You have to copy them to the directory of your installation (e.g. where start_xwiki.sh script is).

Troubleshooting

Dealing with Out-of-Memory Errors

If you get an OutOfMemoryError then increase the maximum heap space by setting the MAVEN_OPTS environment variable. For example for building XWiki Products - Curriki - Database one seems to need MAVEN_OPTS=-Xmx100m or higher.

Error in Windows installer build: Windres error

Launch4J is using a binary called Windres which requires that the environment is NOT set to use UTF-8. If you get the following error check your LANG environment variable:

Embedded error: net.sf.launch4j.BuilderException: net.sf.launch4j.ExecException: Exec failed(1): /var/tmp/installers/windows/target/dependency/bin/windres --preprocessor=cat -J rc -O coff -F pe-i386 /tmp/launch4j10018rc /tmp/launch4j10019o

Building behind a proxy

If you are connecting to Internet through a proxy then you need to modify your settings.xml file so that Maven knows it.

<settings>
 <proxies>
   <proxy>
     <active>true</active>
     <protocol>http</protocol>
     <host>host</host>
     <port>port</port>
     <username>uname</username>
     <password>password</password>
   </proxy>
 </proxies>
 <!--other tags-->
</settings>

Failure to build xwiki-rendering-macro-groovy module

If you're on Mac and want to run XWiki Enterprise on the JDK 1.5 but have JDK 1.6 installed then the Groovy macro in the new rendering won't build fine and you'll get some exceptions. This is because of a Mac problem: When Java 1.6.0 is installed in OSX 10.5 it overwrites /System/Library/Java/Extensions/AppleScriptEngine.jar with a 1.6 specific one. When the ScriptEngineManager gets initialized if finds it and tries to use it. However it only works on JDK 1.6. Moving both AppleScriptEngine.jar and libAppleScriptEngine.jnilib somewhere else fixes the problem.

Build Tools

We have developed some build tools in the form of Maven plugins. They are located in the platform/xwiki-tools/ directory. Here are descriptions of some of them:

  • xwiki-packager-plugin: Takes XWiki pages defined as XML files and located in src/main/resources/ and load them into a database (defined by src/main/packager/hibernate.cfg.xml). An example usage is available here.

Building in Eclipse

Building in IntelliJ IDEA

Tags:
Created by VincentMassol on 2007/11/26 17:16
Last modified by Caleb James DeLisle on 2010/03/07 22:56
This wiki is licensed under a Creative Commons license
2.2.1.27354