Development Practices

XWiki development follows several rules listed below. If you're a Committer or if you simply wish to contribute please take the time to read them as you're expected to follow these rules.

• General Development Flow
• Automated Builds
• Continuous Integration
• Release Manager Role
• Build Manager Role
• Versioning & Release Practices
• Copyright header in source files
  • For Java files
  • For XML files
  • For Shell scripts and Properties files
  • For Bat files
• Coding Conventions
• File encoding
• Creating a JIRA project
• JIRA Best Practices
  • Rule: Always put a JIRA issue reference in commit messages
  • Rule: Don't create unnecessary issues
  • Rule: Use nice user-friendly titles
  • Rule: Don't set a "Fix For" field for invalid issues
  • Rule: Close issues that XWiki committers don't plan to implement
• Documentation Best Practices
• Using bird names for Skins
• SCM Practices
  • Applying a contributor's patch
• Back-end  Development Practices
  • Component Development
  • Backward Compatibility
    • Deprecation Rules
    • @Unstable Annotation
  • Configuration Property Naming
  • Translation Property Naming
  • Translation Property Deprecations
  • Migrating away from the Old Core
• Front-end Development Practices
  • JavaScript Best Practices
    • External libraries
    • XWiki Namespacing
    • Backward compatibility and deprecation
• Bug Reporting
• XWiki.org Rules
  • Don't remove important pages when they are moved
  • Don't use URLs for links to *.xwiki.org
• XWiki Days
• Application Development
• Build Best Practices
• Top Level Extensions

General Development Flow

• Whenever we start working on a topic (a sufficiently important one), we send a proposal email to the list to explain what we're going to work on, to gather feedback and to get agreement.
• In the proposal email we propose a roadmap, i.e. dates that are in sync with project roadmaps (for example, say: "this feature is planned to be finished for XE 1.7M2).
• Once we get the agreement, we create a JIRA issue and assign oneself (we don't forget to set the fix for version)
• If it takes long (say more than 1 week) we regularly publish feedback on the progress and possibly revise the dates if need be. We should absolutely warn if there's a risk of not delivering as planned. This is crucial.
• Commits should be done regularly and more than once per week (the more the better, ideally several times per day).

The goals are several:

• Provide visibility to others
• Ensure agreement on the work being done (it's too stupid to do a lot of work and only find when it's finished that it wasn't the right way to do it and it has to be all redone again)
• Allow us to have product roadmaps
• Provide the ability to help someone if he's lagging behind or has issues

Automated Builds

XWiki has an automated build system and developers are asked to use it on their local machines to prove that their changes work before committing them to the Source Repository. See the Building page for more details.

Continuous Integration

See the Continuous Build page.

Release Manager Role

Role definition:

• In charge of performing a Release of XWiki, usually XWiki Commons/Rendering/Platform/XE/XEM.
• Is responsible to ensure that the dates defined in the Roadmap are achieved
• Needs to coordinate with developers so that they are ready on the release day, this means warning several days ahead and shepherding them to finish their stuff on time or push stuff to the next release
• Needs to follow the Release Process.
• There's a Release Manager Roster to keep track of past Build Managers (and future ones).

Build Manager Role

Role definition:

• Every week we have a different Build Manager chosen amongst the Committers
• The Build Manager has the responsibility to get the build fixed ASAP whenever it's failing. His priority #1 during the week becomes monitoring and having the Build in working order (i.e. no failure)
• By "Build" we mean the CI Build on http://ci.xwiki.org and by "failing" we mean anything that makes the build fail: tests, compilation, clirr, etc.
• In order to fix build issues the Build Manager has several possibilities:
  • find out who caused the build to break and ask that person to fix it. That person cannot refuse that and must consider it his/her priority to fix it (or rollback the change that caused the build to fail)
  • rollback the issue that caused the build to fail
  • fix it himself/herself
  • find someone knowledgable in the failing domain and get him/her to fix the build.
• There's a Build Manager Roster to log past Build Managers (and possibly future ones if some have expressed the wish to be the Build Manager for a specific week).
• At the end of the Week the Build Manager must find the next Build Manager and hand over his duty. If he doesn't find anyone else, he remains the Build Manager. Note that for fairness the oldest Build Managers on the Roster need to accept to be the next Build Manager.
• All committers must perform this duty and take turns

Some examples of things that break the build and for which we need someone "owning" the build to fix them:

• Agents stop working. For ex some morning there were 2 jobs stuck because of a failure to start Firefox. This happens from time to time. Someone needs to investigate and at the very least kill the job to free the agent.
• New versions of Jenkins fixing bugs. Someone needs to check and upgrade the build when a new version has interesting stuff for us and when it fixes some of our issues.
• The most important ones: flickering tests. Of course we always test locally before committing and we even check that it works on Jenkins. But flickering don't fail immediately, they might run fine for 50 or 100 iterations and suddenly start to fail. It's not always easy to find who's the culprit on flickering tests.
• Various issues with filesystem locks, permissions on agents, memory settings, number of allowed open files, etc that make the build fail

Versioning & Release Practices

Check the Versioning and Release Practices page.

Copyright header in source files

All files (including configuration files) must have the following copyright statement.

For Java files

For XML files

For Shell scripts and Properties files

For Bat files

Coding Conventions

See the Code Style page.

File encoding

The following rules must be taken into account when the need for writing non-ASCII content in the XWiki sources arises:

• All Java source files must contain only ASCII chars, Unicode escapes inside strings when needed, and XML entities in javadocs. Since we don't use @author tags, this should not be a problem.
• All translation files must contain only ASCII chars and Unicode escapes (stronger than the Java properties specification).
• All wiki documents sources must be stored in UTF-8.
• Other XML files should always specify their encoding in the <?xml> header, and it should be UTF-8 as often as possible.
• All other textual resources must be stored in UTF-8, minimizing the use of non-ASCII chars.

Currently, only the XML rules are broken, but this is not a serious problem since the XML reader can detect and use the charset/encoding specified in the XML header.

Creating a JIRA project

When creating a new JIRA project follow these steps:

• Create a "JIRA Classic" project type
• Put as Project Lead the person requesting the project
• Make sure to fill the URL part. Ideally it should point to documentation on extensions.xwiki.org. Failing that it should point to the GitHub project's page.
• Make sure to choose the proper "Category". For Contrib projects, it's "XWiki Contributed Projects".
• Set Workflows to use the "XWiki Workflow Scheme"
• Set Screens to use the "Basic Issue Creation Scheme"
• Set Fields to use the "XWiki Open Field Configuration Scheme"
• Create the appropriate versions. If the project has already had releases, recreate the versions and release them in JIRA with the correct release dates
• Make sure that the default assignee is "Unassigned" and that it's not the Project Lead by default
• Set Permissions to use the "XWiki Open" scheme
• Set Issue Security to use the "XWiki" scheme
• Set Notifications to use the "XWiki Notification Scheme"

Note: Ideally we would use a custom JIRA project template that has all this set up but it seems it's complex to create a template in JIRA and that it requires Java coding.

JIRA Best Practices

Here are some rules on how to use JIRA for XWiki projects.

Rule: Always put a JIRA issue reference in commit messages

The rationale behind this:

• One consistent way to manage all the work to be done on the XWiki project. It also means there's no unaccounted work, meaning anyone can go to JIRA and query it and see what everyone has been working on.
• Automated release notes/change logs. When we release a version we can simply do a JIRA extract and it'll give the full change log of what happened. This is really important for our users to see what has been done when in XWiki.
• Traceability. When you use the JIRA issue number in your commit, the JIRA DVCS Connector Plugin can show the modified files directly from JIRA. This is quite useful later on, when someone is looking at a JIRA issue and wants to see what was modified. In addition Fisheye is also integrated with JIRA and when you browse the source repository you can see the JIRA issue associated with files.

Of course this shouldn't be done for any trivial things like adding a small javadoc, renaming a single variable, cosmetic changes, ignore files, etc.

JIRA issues can be marked "confidential", e.g. to hide the details of a security issue until it is fixed. Such issues should also be referenced in the commit message. However, avoid exposing the details of the issue in the commit message (some time will pass until the code is released) by choosing a more neutral description.

The general strategy goes like this:

• When you plan to work on something, create a JIRA issue and assign it to yourself or simply assign an existing JIRA issue to yourself if one already exists
• Implement it
• Commit it with the JIRA issue number in the commit message
• Close the JIRA issue

• Another acceptable variation is:
  • Implement something
  • When you want to commit it you realize you don't have any JIRA issue to put in the commit message so don't commit.
  • Create the JIRA issue
  • Commit with the JIRA number in the commit message

Rule: Don't create unnecessary issues

• If you want to know whether you should create a JIRA issue or not, ask yourself the question: "is my change going to affect any user or any extension developer in any way"? If the answer is yes then you must create a JIRA issue
• If you're fixing something related to build then it's not mandatory to create an issue. However if you do make sure you use the special component usually labeled "Development Issues only" in our JIRA projects. We're excluding issues in this category from our release notes.
• Note that if you're upgrading a third-party dependency used by XWiki at runtime then you must create a JIRA issue using the special component usually labeled "Dependency Upgrades" and this needs to make it in the Release notes since it impacts XWiki users. If you're upgrading a third-party dependency only used at build time (for example the Selenium dependency) then you should use the "Development Issues only" component.
• If you're fixing an issue related to code that's been introduced in the current version being developed, you shouldn't create a JIRA issue. Actually if you did you'd be hard-pressed to find the correct "Affects version" to use! . Instead use the existing JIRA that caused the issue in your commit.

Rule: Use nice user-friendly titles

When you create a JIRA issue always take the time to put a nice title for the issue. The title must be understandable by a user so don't describe the issue technically but rather in what way it affects users. For example don't say "Fix the addXXX API to return an Array List" but rather "Allow creating several pages at once".

Rule: Don't set a "Fix For" field for invalid issues

Issues that are closed with "Won't fix", "Duplicate", "Cannot reproduce" or "Incomplete" must have a "Fix For" field set to "unknown" so that they don't appear in the Release Notes, as JIRA doesn't make the distinction visible and this causes confusion.

Rule: Close issues that XWiki committers don't plan to implement

• We close issues that we know we won't fix (using a "Fix For" value of "Duplicate", "Incomplete", "Cannot Reproduce" or "Won't Fix"). We might not want to fix them for several reasons but one reason is that the issue is for an old XWiki project version.
  • This is especially true since XWiki committers are only officially supporting a few branches. Of course, any committer wanting to
    fix any old issue is free to do it.
• Leaving these issues open is not a good idea since:
  • It sends the wrong signals that we're going to fix the issue
  • The issue can still be found with a search even if closed
  • When we close it then the reporter can explain why it's so important for him/her, or not. Which we wouldn't know if we hadn't closed it
  • Contributors can reopen issues and attach a patch or just create a new issue
  • It means that for all other issues we plan to fix them at some point

Documentation Best Practices

The strategy is that when closing an issue in JIRA we make sure that there's documentation for it on xwiki.org (reference documentation) and on the Release Notes for the version corresponding to the JIRA fixfor field. This is done by filling the 2 corresponding fields in JIRA on the issue that is being closed. This allows two things:

• Have up to date documentation on xwiki.org
• Build Release Notes progressively so that when we perform the Release we don't have to wait a few more days to have everyone try to remember all the stuff they've done for the release...

Using bird names for Skins

We have decided to name XWiki official skins using bird names. We are currently renaming the existing skins. Any new skin must use a bird name. It's recommended to send a proposal to the list whenever a new skin has to be named.

Here are some potential ideas for future names:

• Dove
• Emu
• Kiwi
• Grasswren
• Condor
• Warbler
• Kestrel
• Redtail
• Pigeon
• Booby
• Sparrow
• Blackbird
• Parrot
• Peacock
• Finch
• Wren
• Crow
• Turkey
• Thunderbird
• Piasa (a local phenomenon, from a primitive painting on the cliffs on the eastern shore of the Mississippi, attributed to the Piasa tribe, which looks rather like a griffin)
• Thunder Chicken
• Kingfisher
• Swallow

SCM Practices

Applying a contributor's patch

Please use the following commit log message template:

This gives credit to the contributor.

Also make sure to add the contributor's name to the Contribution page.

This is important to give proper recognition to our contributors.

Back-end  Development Practices

Component Development

Read the Component Module documentation for details on how to write components.

Backward Compatibility

We pay a lot of attention to backward compatibility. This is why we're using the CLIRR Maven plugin in our builds to ensure we don't break public APIs. 

Since it takes time to stabilize an API we've introduced an annotation named @Unstable (see below).

We follow 2 steps when it comes to deprecating code:

• First step: We start by deprecating it using the @deprecate and @Deprecated Javadoc and java annotations. The code remains where it was before the deprecation happens.
• Second step: When our own code doesn't use any of the deprecated APIs anymore then move the code to a legacy module. This can be done as soon as the deprecation is added.

This has the following advantages:

• Our code remains clean of old deprecated APIs
• Deprecated code is cleanly separated from new ways of doing things
• When new users download the our code they see the new ways (same for javadoc generation)
• Users wanting to use the old APIs can still do so
• We never remove APIs from the legacy modules by default. However if we really need to do so (for some technical reason for example), we do it on a case by case basis with a VOTE.

Deprecation Rules

For details and rationale about those rules below see this thread.

• Rule: Location of Legacy modules
  • Each Git repository needing legacy modules provides a *-legacy module for holding legacy modules. For example:
    • For Commons, xwiki-commons-core/xwiki-commons-legacy/
    • For Platform, xwiki-platform-core/xwiki-platform-legacy/
    • For Rendering, xwiki-rendering-legacy/
• Rule: Use AspectJ to move deprecated APIs to Legacy modules
  • This idea was found in this blog post.
  • Here's an example of such an Aspect.
• Rule: Legacy modules replace modules from where they come from
  • Each Legacy module replace the non-legacy module it corresponds to. This means that the user must have only 1 JAR for a given module: either its legacy version or it's non-legacy version but should never have both.
• Rule: Annotate code with version to show deprecations
  • Use both @Deprecated annotation and the @deprecated javadoc tag and specify the version when the deprecation was added. For example:
    /**
     * @param time the time in milliseconds
     * @return the time delta in milliseconds between the current date and the time passed as parameter
     * @deprecated replaced by {@link com.xpn.xwiki.api.Util#getTimeDelta(long)} since 1.3M2
     */
    @Deprecated
    public int XWiki.getTimeDelta(long time)
    {
       return this.util.getTimeDelta(time);
    }

@Unstable Annotation

This annotation can (and should) be used by developers whenever new code is introduced. This annotation means that the code is subject to change at any time (in which case it'll appear in the CLIRR reports but with the explanation that it's an Unstable API). From a user point of view, it means they should use classes or methods annotated with the @Unstable annotation with caution. It means they need to be prepared to modify their code. Alternatively they can not use classes/methods annotated with @Unstable and wait for them to come out of unstability.

In order to prevent code to remain annotated with @Unstable forever, we've defined some rule;

• Any code annotated with @Unstable can only remain in this state for a maximum of 1 full cycle. For example if a unstable API is introduced in 4.4, it'll have to come out of unstability before 6.0M1 is released (thus having 1 full cycle, i.e. the 5.x cycle). This means that the worse that can happen is for an unstable API added in N.1 which will have to be removed before N+2 Milestone 1 is released.
• However, the previous rule is only a maximum and developers are encouraged to remove the unstable annotation whenever they feel that the API should come out of unstability. When this happens the standard deprecation mechanism then kicks in.

Configuration Property Naming

XWiki uses 2 configuration files at the moment:

• An old one, named xwiki.cfg for old core code and which eventually will go away when all code will have been rewritten using component and extracted as separate modules
• A more recent one that must be used for new configuration properties required by new code, named xwiki.properties

The naming rule for xwiki.properties is:

• Use <module>.<propertyName>. Ex: rendering.linkLabelFormat
• For submodules use <module>.<submodule>.<propertyName>. Ex: rendering.macro.velocity.filter
  • Note: Unfortunately you'll find some properties not following this rule in xwiki.properties. This is because we've been bad in following this rule and doing code review to ensure it was followed. It was also voted on the mailing list but not documented here in the past.
• Use camelcase for the property name itself. ex: linkLabelFormat

Translation Property Naming

XWiki uses some properties file containing translation resources. The format for translation properties is:

 where:

• <short top level project name> = top level project name without the xwiki- prefix, for example: commons, rendering, platform, enterprise, manager, etc
• <short module name> = the name of the Maven module without the <short top level project name> prefix, for example: oldcore, scheduler, activitystream, etc
• <propertyName> = the name of the property using camel case, for example updateJobClassCommitComment

Translation Property Deprecations

• When deleting a key, it should be moved to the deprecated section at the end of the file, linked to the first version in which it started to be deprecated
• When renaming a key, it should be moved to the same deprecated section and a comment should be added with the following syntax:
  #@deprecated new.key.name
  old.key.name=Some translation

Migrating away from the Old Core

Starting in 2006 we've started to work on splitting the code that's currently located in the xwiki-platform-oldcore module into various modules, each one specific to a given domain. Before 2006 the whole of XWiki was located in that oldcore. The work isn't over and the current strategy is the following:

• Continue extracting code from oldcore and put it in its own domain modules
• We allow new modules to depend on oldcore provided that oldcore itself doesn't depend on these new modules obviously (as otherwise it would create a cyclic dependency). When that happen it usually means we need to move the code in oldcore that uses thee new module outside of oldcore in an existing module or in a new one.
• At some point all that should remain in oldcore is the old Model itself.
• We need to continue the work we started about writing the New Model and put it in some new module, at which point we'll need to migrate our code to use the new Model progressively and then the last step will be to remove oldcore altogether.

Front-end Development Practices

JavaScript Best Practices

All javascript should be in the <head> of the document.
There are a variety of practices which lead to less useful more error prone pages which sometimes violate WCAG standards, these are almost impossible if all javascript in inserted into the head.
See this email thread

Avoid using javascript in attributes such as onload, onmouseover, etc. Doing this can cause problems with event handlers and is sometimes WCAG invalid.

Users should be able to do anything without javascript that they can with it.
Javascript should only exist to make your web experience more enjoyable.

External libraries

XWiki bundles several libraries to ease the development of front-end components. Amongst them are:

• Prototype.js (DOM manipulation, AJAX library, OOP-style classes)
• Smartclient (Rich AJAX widgets)
• Scriptaculous (Drag and drop, animation framework, AJAX controls, DOM utilities and unit testing. Add-on for the Prototype library.)

For an exhaustive list, including XWiki internal libraries, see FrontendResources

Front-end components should rely on components provided by those libraries as much as possible.

The introduction of new libraries should go through a vote on XWiki devs list. Only libraries which bring functionalities that none of the already provided external or internal libraries do are likely to be considered. 

XWiki Namespacing

All XWiki's JavaScript code should be under the XWiki namespace. The preferred way to namespace a module's code is to use the Module Pattern, in particular loose augmentation and sub-modules.

Methods, classes and properties that have sense only in one module should be created under a module object of the XWiki object. 

Example:

Methods, classes and properties that have sense in all XWiki pages can be considered generic and be create directly under the XWiki object.

Example:

Backward compatibility and deprecation

As for back-end development, we should take care of backward compatibility. There are applications developed on top of XWiki that use their own JavaScript APIs. The rule for deprecating a JavaScript API (be it an object or a method of an object) is to move or write code to maintain compatibility in the compatibility.js file under the xwiki folder. We should always wrap deprecated code to log its usages. This will make the life of applications developers much easier. 

Example of deprecation code:

Bug Reporting

The bug reporting system resides at http://jira.xwiki.org/

XWiki.org Rules

Some rules to follow when making modifications on xwiki.org.

Don't remove important pages when they are moved

Rationale: Users will save links and when they navigate to them later on these links will be broken.
Solution: Add a redirect script to redirect to the new page
Implementation: Use this script:

Don't use URLs for links to *.xwiki.org

Rationale: If we change the URL format or the domain the links will be broken
Solution: Use the wiki:Space.Name notation

XWiki Days

See XWiki Days.

Application Development

See Application Development Best Practices.

Build Best Practices

• For the general directory structure, see SourceRepository.
• Directory names: We're currently using the a directory name corresponding to the Maven artifactId. For example: xwiki-platform-watchlist-ui for a pom.xml having
  <artifactId>xwiki-platform-watchlist-ui</artifactId>

  Note that this scheme is causing some issues on Windows since it generates long path names and Windows usually supports only 255 characters. It's possible that we'll change this in the future... or not, since you can use Cygwin on Windows to have longer path names.
• Maven groupIds: org.xwiki.<short name of top level project>. For example org.xwiki.commons for XWiki Commons, org.xwiki.rendering for XWiki Rendering, org.xwiki.platform for XWiki Platform and org.xwiki.enterprise for XWiki Enterprise
• Maven artifactIds: xwiki-<short name of top level project this module belongs to>-<module name>-<qualifiers>. For example:xwiki-platform-watchlist-ui. There are some conventions used: -api for modules providing APIs, -ui for modules generating XARs, -test for functional test parent POM, -test-pageobjects for functional test Page Objects modules, -test-tests for modules containing and executing the functional tests.

Top Level Extensions

Rationale/Need:

• Be able to extract some apps from xwiki-contrib that the XWiki Dev Team would like to maintain. Example: File Manager app developed by Marius when it’ll have had some releases and tests (if it doesn’t have some already!), GitHub Stats app used on xwiki.org, Meeting Manager App, Forum App, etc 
• Be able to extract some extensions currently located in xwiki-platform but not released with XE so that they can have a different release cycle (examples: FAQ app, IRCBot extension, JIRA macro, etc). Having different release cycle allow to release new versions quicker to our users (bug fixes, new features). 

Governance:

• Extensions are VOTEd in on a case by case basis. 
• Each voted extension has its own Git Repository in the “xwiki” organization (so that each extension can be released independently of each other). 
• When moving an extension either from xwiki-contrib or from xwiki-platform, keep its Git history as much as possible or simply donate the repo to the xwiki organization. 
• FTM extensions bundled by default with XE still remain in XWiki Commons/Rendering/Platform/Enterprise. 
• The Git repository name must be of the form xwiki-<short project name>. <short project name> must be part of the VOTE. 
• All XWiki Development rules apply 
• Each extension has a Release Manager defined and he’s responsible for defining its own Roadmap/Release notes (if need be), on the extension page on e.x.o and perform the releases or ensure the extension is released regularly when there are changes. 
• Each extension must follow these criteria for being VOTEd:
  • A Release Manager needs to be defined in the proposal 
  • The extension must have had several releases already (i.e. someone wanting to propose a new extensions that doesn’t exist would start in xwiki-contrib for ex and prove that his extension works and is useful by doing several releases and creating the pages on e.x.o) 
  • It must follow our best practices (coding practices, tests, etc) and follow the apps best practices (for apps).
  • It must have one or several integration or functional tests (for apps) to prove that it works. This allows to prove the app continues working when XWiki progresses 
  • The main contributors of the extensions must agree about the move. If they have the “level" to be an xwiki dev committer then they should be voted in. If not then either they’re ok to send Pull Requests or the extension should not be moved. 
• If an extension ceases to work or if its quality becomes too low, it can be moved to xwiki-contrib with a VOTE 
• We create one JIRA project per extension 
• We create a new JIRA Category called “XWiki Extensions” 
• We put the extensions in our CI at http://ci.xwiki.org 
• The Java package must follow the same rule as for XWiki Platform, i.e. org.xwiki.<short project name>. Exceptions would need to be discussed. 
• The group id for extensions having their own repo must be org.xwiki.<short project name>. The <short project name> needs to be part of the VOTE when proposing a new extensions.