Changes for page Building XWiki from sources

Last modified by Vincent Massol on 2024/04/08 16:22

From 58.1 to 59.1 From 60.1 to 60.2

From version 59.1

edited by Vincent Massol
on 2012/03/31 14:13

Change comment: There is no comment for this version

To version 60.1

edited by Vincent Massol
on 2012/04/03 10:25

Change comment: Update

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

@@ -1,10 +1,6 @@
  {{box cssClass="floatinginfobox" title="**Contents**"}}{{toc/}}{{/box}}
  {{error}}
--31 March 2012: This page contains lots of obsolete information and needs to be updated.
--{{/error}}
--
--{{error}}
  Make sure you install Maven 3.0 or greater. It might build fine using earlier versions but we're not supporting them and all the XWiki committers and our CI server are running 3.0 and greater.
  {{/error}}
@@ -12,23 +12,21 @@
  Before you start it's good to have some minimal understanding of [[how Maven works>>http://maven.apache.org/users/index.html]]. You probably won't need it if everything goes fine but you'll need that knowledge if something breaks! ;-)
--The first thing to understand is the [[directory structure used>>SourceRepository]].
++The first thing to understand is the [[directory structure we use>>SourceRepository]].
  = Checking out the sources =
--Use your favorite SCM client to [[check out the sources>>SourceRepository]].
++Use your favorite SCM client to [[check out the Project>>SourceRepository]] you wish to build.
--You can check out the whole source tree or only a single module you wish to build. Maven is powerful enough in that it'll always try to download the required dependencies from the remote repositories you have defined, so that you don't have to build the whole project from sources.
--
  = Installing Maven =
--* Install [[Maven 3.0 or greater>>http://maven.apache.org]]. Create an M2_HOME environment variable pointing to where you have installed Maven. Add //M2_HOME/bin// to your PATH environment variable.(((
++* Install [[Maven 3.0 or greater>>http://maven.apache.org]]. Create an ##M2_HOME## environment variable pointing to where you have installed Maven. Add ##M2_HOME/bin## to your PATH environment variable.(((
  {{info}}For more information on how to install Maven or how to use it check this [[excellent Maven book>>http://www.sonatype.com/book/reference/public-book.html]].{{/info}}
  )))
--* Make sure you give Maven's JVM enough memory. XWiki's build uses Aspects and GWT compilers which need lots of memory. A good value is to give the JVM 600MB. To do that set an environment variable named ##MAVEN_OPTS##. For example on Unix add the following to your shell startup script:(((
--{{code language="none"}}MAVEN_OPTS="-Xmx500m -XX:MaxPermSize=256m"{{/code}}
--))) (depending on your shell you might need to prefix the variable with ##export##)
--* Create a ##~~/.m2/settings.xml## file with the XWiki custom remote repository defined as shown below. If you are on Windows, create the directory ##.m2## in your home directory, e.g. ##C:\Documents and Settings\Administrator\.m2##. If you cannot do this from Windows Explorer, open a commandline and use ##md .m2##.(((
++* Make sure you give Maven's JVM enough memory. XWiki's build uses Aspects and GWT compilers which need lots of memory. A good value is to give the JVM 512MB. To do that set an environment variable named ##MAVEN_OPTS##. For example on Unix add the following to your shell startup script (depending on your shell you might need to prefix the variable with ##export##):(((
++{{code language="none"}}MAVEN_OPTS="-Xmx512m -XX:MaxPermSize=256m"{{/code}}
++)))
++* Create a ##~~/.m2/settings.xml## file with the XWiki custom remote repository defined as shown below. If you are on Windows, create the directory ##.m2## in your home directory, e.g. ##C:\Documents and Settings\Administrator\.m2##. If you cannot do this from Windows Explorer, open a command line and use ##md .m2##.(((
  {{code language="xml"}}
  <settings>
@@ -91,152 +91,94 @@
  </settings>
  {{/code}}
--{{info}}Depending on your geographical location, you should use closer maven repositories to speed up your build process. For more information refer [[Maven Mirror Settings Guide>>http://maven.apache.org/guides/mini/guide-mirror-settings.html]].{{/info}}
++{{info}}Depending on your geographical location, you should use closer Maven repositories to speed up your build process. For more information refer [[Maven Mirror Settings Guide>>http://maven.apache.org/guides/mini/guide-mirror-settings.html]].{{/info}}
  )))
  = Building with Maven =
--Projects are directories found under the XWiki SCM root. For example: platform, enterprise, manager, etc.
++Now that you have installed Maven and checked out the [[Projects>>SourceRepository#HTopLevelProjects]] you wanted you'll want to build them. To build a project with Maven just issue the command ##mvn install##.
--To build them, check them out and run ##mvn install##.
++Each project is made of modules. Thanks to XWiki's [[Continuous Integration setup>>DevelopmentPractices#HContinuousIntegration]] you can check out and build any single module (and its children) you wish without having to rebuild the other modules from source. Maven will pick the latest version of your module's dependencies from [[XWiki's Maven remote repositories>>http://maven.xwiki.org]]. Of course if you have uncommitted changes in a dependent module, you'll want to build that module before. The build result is placed in two places:
++* Your [[local maven repository>>http://maven.apache.org/guides/introduction/introduction-to-repositories.html]], for making the module available to other projects or modules using maven (even other XWiki modules take the needed libraries from this repository, and not directly from a "neighbor" directory)
++* In a subdirectory of that module, called ##target##.
--Each project is made of modules. Thanks to XWiki's [[Continuous Integration setup>>Community.DevelopmentPractices#HContinuousIntegration]] you can check out and build any single module (and its children) you wish without having to rebuild the other modules from source. Maven will pick the latest version of your module's dependencies from [[XWiki's Maven remote repositories>>http://maven.xwiki.org]]. Of course if you have uncommitted changes in a dependent module, you'll want to build that module before. The build result is placed in two places:
--* the [[local maven repository>>http://maven.apache.org/guides/introduction/introduction-to-repositories.html]], for making the module available to other projects or modules using maven (even other XWiki modules take the needed libraries from this repository, and not directly from a "neighbor" directory)
--* and in a subdirectory of that module, called ##target##.
++{{info}}
++Now some of the modules built by XWiki are very large (in the 40MB+ size) and sometimes you don't absolutely need the latest version to get downloaded as it'll take too long. In that case here's a tip: run Maven in 'no-snapshot-update' mode with ##mvn install -nsu##.
++{{/info}}
--{{info}}Now some of the modules built by XWiki are very large (in the 40MB+ size) and sometimes you don't absolutely need the latest version to get downloaded as it'll take too long. In that case here's a tip: run Maven in 'no-snapshot-update' mode with ##mvn install -nsu##.{{/info}}
++{{warning}}
++You might encounter an ##java.lang.OutOfMemoryError## during ##aspectj:compile##. You must increase the maximum memory used by maven, by setting ##MAVEN_OPTS=-Xmx512m## (or a larger amount of memory if needed).
++{{/warning}}
--{{warning}}You might encounter an ##java.lang.OutOfMemoryError## during ##aspectj:compile##. You must increase the maximum memory used by maven, by setting ##MAVEN_OPTS=-Xmx512m## (or a different amount of memory).{{/warning}}
--
  You'll find below some common build tasks.
--== Building for a specific database ==
++== Using Profiles ==
--By default XWiki's Maven build will use Hypersonic SQL as the default database. If you wish to execute the build using a different database, you'll need to specify the correct database profile (##-P<profile name>##). Right now we support 4 profiles:
--* ##hsqldb##: Hypersonic SQL. This is the default
--* ##mysql##: MySQL
--* ##pgsql##: PostGreSQL
--* ##derby##: Apache Derby
++Project builds define several [[Maven Profiles>>http://maven.apache.org/guides/introduction/introduction-to-profiles.html]]. We've standardized their names across Projects.
--For example to build a module for MySQL type ##mvn install -Pmysql## in that module's directory.
--
--{{warning}}Hypersonic and Derby are embedded database so you won't need anything setup to execute the build for them. However for MySQL and PostGreSQL you'll need to have the database installed, set up and running before you can run the build with their profiles.
--
--A simple way to do this is to define the necessary properties within ##{HOME}/.m2/settings.xml## as explained above when configuring the maven mirrors. Check the property names required by looking at the corresponding profiles section in the [[platform's top level pom.xml file>>https://github.com/xwiki/xwiki-platform/blob/master/pom.xml]]
++Here are the most common ones:
++* ##legacy##: Includes the [[legacy modules>>DevelopmentPractices#HDeprecation26LegacyStrategy]] in the build.
++* ##integration-tests##: Executes integration and functional tests in the build.
++* ##jetty##, ##glassfish##: Run the build for the specified container (##jetty## is the default when not specified).
++* ##hsqldb## (Hypersonic SQL), ##mysql## (MySQL), ##pgsql## (PostgreSQL), ##derby## (Derby): Run the build for the specified database (##hsqldb## is the default when not specified).(((
++{{warning}}
++Hypersonic and Derby are embedded database so you won't need anything setup to execute the build for them. However for MySQL and PostgreSQL you'll need to have the database installed, set up and running before you can run the build with their profiles.
  {{/warning}}
++)))
++* ##firefox## (Firefox), ##iexplore## (Internet Explorer), ##chrome## (Google Chrome), ##otherbrowser## (need to define ##browserPath## property to use that one): Run the functional tests on the specified browser (##firefox## is the default when not specified).
++* ##dev##: only build wysiwyg for English/Firefox. Very useful to speed up the build of XWiki Platform
--== Building on Windows ==
++More "exotic" ones:
++* ##clover##: A profile to use when [[generating Test Coverage reports>>Testing#HTestCoverage]] with [[Clover>>http://www.atlassian.com/software/clover/overview]]. This profile skips execution of Checkstyle, CLIRR and Enforcer plugin checks to speed up the coverage generation and because of potential conflicts between tools.
++* ##release##: Used by the Maven Release plugin when releasing XWiki. It generates Javadoc and Source artifacts. It also checks that the Java version used is JDK6, that the Javadoc exists and it signs artifacts using GPG (this is a good practice and a [[requirement for being able to upload some of our artifacts to the Maven Central Repository>>http://maven.apache.org/guides/mini/guide-central-repository-upload.html]]).
++* ##m2e##: For [[m2eclipse>>http://www.eclipse.org/m2e/]] users. It sets the Eclipse output directory to ##target-eclipse## (instead of ##target##) to prevent race conditions between Maven within Eclipse and Maven on the command line.
++* ##unix##, ##mac##, ##windows##: These profiles are automatically activated depending on the OS the build is running on. These profiles are useful for the Installers and for functional tests to decide how to start XWiki.
++* ##macprofiler##, ##winprofiler##: Start [[XWiki for Profiling>>Profiling]] (they require that you set a ##profilePath## property in your ##settings.xml## or on the command line)
++* ##gwt-test-manual##: allow running GWT unit tests manually
++* ##debug##: [[weave debug aspects for profiling>>Profiling]] and remove JS minification.
--To build on Windows, you should specify the ##windows## profile (##-Pwindows##).
++For example, if you use to include legacy modules and run all integration and functional tests and use Jetty/HSQLDB, you would use:
--== Building XWiki Enterprise ==
++{{code}}
++mvn clean install -Plegacy,integration-tests,hsqldb,jetty
++{{/code}}
--Go in ##enterprise/## and type ##mvn install##. It'll build it with a default config for HSQL DB. After the build is finished you'll find the standalone zip in ##enterprise/distribution/jetty/hsqldb/target/##.
++== Relationship between XWiki Enterprise and XWiki Platform ==
--{{info}}If you wish to build all the modules required for XWiki Enterprise and not the ones required by other products you can go at the top level and type ##mvn install##"{{/info}}
--
--{{info}}The functional tests are not executed by default as the take a long time to execute and are meant mostly to be executed by the Continuous Integration tool. However if you wish to run them you can execute ##mvn install -Pxe,ci##{{/info}}
--
--{{info}}The installer builds are not executed by default (they are executed on the Continuous Integration server though). If you wish to run them you can execute ##mvn install -Pxe,ci## or better, go in ##xwiki-product-enterprise/installers## and run ##mvn install##.{{/info}}
--
--== The Relationship between XWiki Enterprise and XWiki Platform ==
--
  XWiki Platform is a generic platform and API for building flexible extensible wikis and web applications. XWiki Enterprise is an implementation of the platform geared toward making a wiki site for teams to share information and synchronize their efforts although it can be and has been used for a wide variety of different use cases like this website.
  What does this mean to the developer?
--If you wish to make changes in the core of XWiki, you are probably going to want to work on XWiki Platform. You can download and compile XWiki Enterprise and while compiling, you will notice Maven downloads a big ~~80MB file, this is the platform. Maven is smart enough to know that you don't have a copy of the platform and will download it from the XWiki Maven repository. Once it has been downloaded the first time, it is stored on your harddrive and you won't have to download it again //until// changes are made in the platform.
++If you wish to make changes in the core of XWiki, you are probably going to want to work on XWiki Platform. You can download and compile XWiki Enterprise and while compiling, you will notice Maven downloads a big 80MB+ file, this is the platform. Maven is smart enough to know that you don't have a copy of the platform and will download it from the XWiki Maven repository. Once it has been downloaded the first time, it is stored on your harddrive and you won't have to download it again //until// changes are made in the platform.
--If you want to work on the core or if you have limited bandwidth, then you are well advised to download the platform. Then you can use the SCM's update command to get the latest copy without downloading the entire platform all over again. If you download the platform and compile it then you can compile XWiki Enterprise, Maven will use your newly compiled copy of the platform rather than downloading it, Maven is smart.
++If you want to work on the core or if you have limited bandwidth, then you are well advised to get the platform sources. Then you can use the SCM's update command to get the latest copy without downloading the entire platform all over again. If you download the platform and compile it then you can compile XWiki Enterprise, Maven will use your newly compiled copy of the platform rather than downloading it, Maven is smart.
--== Building XWiki Platform ==
++= Automatic Checks =
--The platform is built much the same way as XWiki Enterprise but it is a large project and compiling can take a long time (over 1/2 hour.)
--One of the reasons building the platform is slow is because the [[WYSIWYG module>>#HBuildingtheWYSIWYGeditor]] builds a version for each supported browser in each language. You can make it only build for English/Firefox by specifying ##**-Pdev**## in the build command.
--**##mvn install -Pdev**##
--You can also build the platform faster if you skip the unit tests by building with the command:
--**##mvn install -Dtest=none -DfailIfNoTests=false##**
--{{warning}}**Caution:** If you have just made a patch, you should probably run the tests since breaking a test is embarrassing.{{/warning}}
--For added safety (especially when you have made changes which might affect the ability of other code to find the libraries it needs) you can compile with:
--**##mvn clean install##**
--and everything will be recompiled.
--Press enter, go get a cup of coffee, and wait.
++The XWiki build performs some automated checks:
++* Style checks with Checkstyle. To skip: ##-Dxwiki.checkstyle.skip=true##.
++* Verify header licenses
++* Backward compatibility checks with [[CLIRR>>http://clirr.sourceforge.net/]]. To skip: ##-Dxwiki.clirr.skip=true##.
++* Enforcer checks (To skip: ##-Dxwiki.enforcer.skip=true##):
++** Verify that all plugins are versions specified
++** Verify the correct Junit artifact is used (##junit-dep## and not ##junit##)
++** Verify we don't use Commons Logging or Log4j (since we use SLF4J)
++** Verify that the Java version is 6 (in the release profile)
++** Verify that Javadoc exist (in the release profile, this is a Maven Central requirement)
++** In Commons:  Verify that Commons modules don't have a dependency on Rendering and Platform modules
++** In Rendering: Verify that Rendering modules don't have a dependency on Platform modules
--== Executing the Automated Functional Tests for XWiki Enterprise ==
++= Tips ==
--Go in ##enterprise/distribution-test## and type ##mvn install##. By default they are executed in Firefox. To run them in a different browser, use a profile. For example to run them in IE, type ##mvn install -Piexplore##.
++== Skipping Tests ==
--See the [[Testing]] page for more details.
++While you should almost never do that, here's a tip for the rare occasions when you'll need to build something fast:
--== Executing the Standard Web WAR quickly in development mode ==
++{{code}}
++mvn install -Dtest=none -DfailIfNoTests=false
++{{/code}}
--{{warning}}
--Update 2010-03-07: The part below is a bit oldish and is probably not working any more. It needs someone to look at it. Please check the alternative below.
++Alternatives are ##mvn install -DskipTests=true## or ##mvn install -Dmaven.test.skip=true##. Refere to the Maven documentation to understand the differences.
--Go in ##platform/web/standard## and type ##mvn install -Pjettyrun##. This will execute it on Jetty with a HSQLDB database located in ##platform/web/standard/database## (we need to improve the build to locate it elsewhere). So if you want to have any content displayed you'll need to put a HSQLDB there or simply import a XAR.
--
--{{info}}If you wish to use another database like MySQL, run ##mvn install -Pjettyrun,mysql##. See the section above on building XWiki for a specific database for more information.{{/info}}
--
--Once the build has started Jetty, open your browser and point it to ##[[http://localhost:8080/xwiki]]##.
--
--To debug in this mode, run ##MAVEN_OPTS='-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n' mvn install -Pjettyrun## and attach you IDE to this process as in [[Debugging#HRemoteDebugging]].
--
--Note that this is a quick way of starting XWiki in development mode. However the real test is to start the XWiki Enterprise standalone distribution to ensure that everything works. This is achieved by running ##mvn install## in ##enterprise/##.
--{{/warning}}
--
--{{info}}Here are some alternative strategies used by XWiki committers:
--
--* [[Debug with Eclipse and m2eclipse>>dev:Community.DebugXEWithEclipse]]
--* Keep a running XE and copy modified files to it. You can even start your container in development mode so that it reloads the copied files automatically.
--* Download the latest XE SNAPSHOT from [[XWiki's Continuous Integration server>>http://maven.xwiki.org/snapshots/]] and unzip it and then start it.{{/info}}
--
--== Building XEclipse ==
--
--{{warning}}You'll need Eclipse 3.3.x (or above) and an associated PDE (Plugin Development Environment) in order to build XEclipse from source.{{/warning}}
--
--This section is about building XEclipse with maven. For instructions on how to build XEclipse from right within the Eclipse IDE, you may refer to [[Building in Eclipse>>dev:Community.BuildingInEclipse]]
--
--=== Building XEclipse as an Eclipse plugin ===
--
--Change directory to ##plugins/org.xwiki.eclipse## and run ##mvn install -DeclipseInstall=$ECLIPSE_DIR##
--
--=== Builing XEclipse as a standalone application ===
--
--You will need the [[pde-maven-plugin>>http://mojo.codehaus.org/pde-maven-plugin/]].
--Once this maven plugin is installed, please refer to ##plugins/org.xwiki.eclipse.rcp/README## for further information about building instructions.
--
--== Building the WYSIWYG editor ==
--
--=== Version 2.2.x and older ===
--
--The WYSIWYG editor's code is split in three modules, all found in ##platform/web## directory:
--
--* ##xwiki-gwt-dom## (DOM API, Selection and Range API)
--* ##xwiki-gwt-user## (Widgets including the RichTextArea)
--* ##wysiwyg## (The editor code, including plugins)
--
--The ##wysiwyg## module depends on the ##xwiki-gwt-dom## and ##xwiki-gwt-user## modules. The ##xwiki-gwt-user## module depends on ##xwiki-gwt-dom## module. After you build a module you have to also build the modules that depend on it for your changes to take effect. To build one of the modules you just have to execute ##mvn clean install## in the corresponding directory. ##xwiki-gwt-dom## and ##xwiki-gwt-user## modules generate jars (so no JavaScript). This jars are required at GWT compile time. The ##wysiwyg## module generates a war that is merged with the war build from ##platform/web/standard##. The WYSIWYG war contains mainly the GWT-compiled JavaScript code and the server side jar.
--
--By default, when you build the ##wysiwyg## module the GWT compiler generates compilation units (permutations) for each major browser and for each supported locales. Because of this the build takes a while and requires a lot of resources (both memory and processor). We cannot afford this during developemnt so we build the editor just for one browser and one locale at a time. We achive this by using the ##-Pdev## maven profile and by specifying the browser and the locale in the ##src/main/resources/com/xpn/xwiki/wysiwyg/WysiwygDev.gwt.xml## module descriptor that is used for developent. This module descriptor inherits ##Wysiwyg.gwt.xml## which is used by default if the ##-Pdev## profile is not activated. If you build the ##wysiwyg## module using the ##-Pdev## profile and you don't change the ##WysiwygDev.gwt.xml## the GWT compiler generates only one permutation for Firefox + English locale.
--
--The JavaScript code is obfuscated by default. Using the ##-Pdev## profile the JavaScript code is kept detailed (for debugging) but its size (5MB at some point, for one permutation) makes it unusable in production. You can control the style of the generated JavaScript code using the ##gwt.style## pom property.
--
--To update the editor of an XWiki Enterprise instance you have to update the ##resources/js/xwiki/wysiwyg/xwe## directory and the ##WEB-INF/lib/xwiki-web-wysiwyg.jar## with the ones from the war generated by the ##wysiwyg## module build. Note that the server side code of the WYSIWYG editor depends on the XWiki platform core and thus the ##xwiki-web-wysiwyg## and ##xwiki-core## jars should have the same version.
--
--=== Version 2.3.x and newer ===
--
--Starting with version 2.3 the ##wysiwyg## module was split in two:
--
--* ##xwiki-gwt-wysiwyg-client##
--* ##xwiki-gwt-wysiwyg-server##
--
--The client side module generates two jars. One is required at GWT compile time and the other, containing classes and interfaces shared between the client side and the server side code, is required at runtime. The server side module generates a war which contains mainly the GWT-compiled JavaScript code and two jars: the shared classes and the server side Java code.
--
--As it used to happen with the ##wysiwyg## module prior to version 2.3, when building the ##xwiki-gwt-wysiwyg-server## module the GWT compiler generates by default all the permutations (browser x locale) and thus requires a lot of resources. You can use the ##-Pdev## maven profile to generate just one permutation, specified in the ##WysiwygDev.gwt.xml## module descriptor. Note that this file is now located in the ##xwiki-gwt-wysiwyg-client## module. As a result, if you want to change the default development permutation (Firefox + English) you have to first build the client side module and then the server side module using the ##-Pdev## profile.
--
--=== Running unit tests in browser ===
--
--Beginning with version 2.2 unit tests are run by default using HtmlUnit (GWT >=2.0) which tries to emulate the browser. Some of the JavaScript API is still not supported (e.g. Selection API is partially implemented) and thus we are forced to disable some of the tests in HtmlUnit mode. Fortunately we can run all the tests in a real browser using the  ##-Pmanual## maven profile. When this profile is active the build stops at the ##integration-test## lifecycle phase and waits for you to point your browser to a given URL. Tests can run in both Java and JavaScript code. You can tweak the test parameters by editing the pom.
--
  = Troubleshooting =
  == Dealing with Out-of-Memory Errors ==
@@ -298,11 +298,11 @@
  ...
  {{/code}}
--It seems there's a bug in Maven 3.0.1. To work around it, use Maven 3.0 instead.
++It seems there's a bug in Maven 3.0.1. To work around it, use Maven 3.0. instead.
  = Build Tools =
--We have developed some build tools in the form of Maven plugins. They are located in the [[##platform/xwiki-tools/## directory>>https://github.com/xwiki/xwiki-platform/tree/master/xwiki-platform-tools]]. Here are descriptions of some of them:
++We have developed some build tools in the form of Maven plugins. They are located in the [[##commons/tools##>>https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-tools]] and [[##platform/xwiki-tools/##>>https://github.com/xwiki/xwiki-platform/tree/master/xwiki-platform-tools]] directories. Here are descriptions of some of them:
  * ##xwiki-packager-plugin##: Takes XWiki pages defined as XML files and located in ##src/main/resources/## and load them into a database (defined by ##src/main/packager/hibernate.cfg.xml##). An example usage is available [[here>>https://github.com/xwiki/xwiki-platform/tree/master/xwiki-platform-tools/xwiki-platform-tool-packager-plugin/src/it/basic-import-export]].
  = Building in Eclipse =

Changes for page Building XWiki from sources

Summary

Details

Quick Links

My Recent Modifications

About

About

Support

Platform

User Guide

Admin Guide

Developer Guide

Projects

XWiki

Extensions

Other

Contribute

Status

Practices

Under the Hood

Get Involved

Get Connected