Development Practices

Last modified by Lucas Charpentier (Sereza7) on 2024/02/12 10:56

XWiki development follows several rules listed below. If you're a Committer or if you simply wish to contribute please take the time to read them as you're expected to follow these rules.

General Development Flow

Servers

Here's the big picture, listing all servers making up the xwiki.org ecosystem and showing all interactions between servers.

developmentProcessV4.png

Roadmap

  • A roadmap proposal email is sent regularly on the devs mailing list, proposing high level topics to work on in the coming XWiki releases.
    • Any committer could do this work but at the moment it's Vincent Massol who's sending the roadmap proposal email. See the info box below to learn more.
  • The idea is to discuss with the XWiki community and especially the XWiki committers what everyone is interested to work on, but also to verify that in the proposed list, there's nothing that the committers think are not going in the good interest of the project. At this point, committers can also propose new items they'd like to work on.
  • Once the roadmap is agreed, or if there are no comments, its content is updated on the Roadmap page
  • Then, each committers having accepted to implement tasks from the roadmap should analyse their tasks, discuss (to cover all topics including functional needs but also cross-topics such as security, risk, performance, and more) with others about them (especially with the person who's proposed the topic), and quickly create a list of JIRA issues, representing the different elements to implement the high level topics. 
  • Note that there can be more JIRA issues than what the committer can take for the release (we currently do monthly releases so committers must take work that take less than a month!)
  • Committers must update the Roadmap page and list the JIRA issue they've decided to implement for the coming release. Thus leaving out for other releases the other JIRAs that they cannot implement for this release. 
  • Committers must then assign themselves on the JIRAs + set the fixVersion field.
  • For complex topics or if they wish to get an agreement from other committers, whenever a committers starts working on a topic, he/she should send a proposal email to the list to explain what he/she's going to work on, to gather feedback and to get agreement. Usually he/she will also create a Design page and link it in the mail.
  • Commits should be done regularly and more than once per week (the more the better, ideally several times per day).
  • Committers are free to create Pull Requests if they want to get some review before pushing the code.

The goals of this process are several:

  • Provide visibility to the community at large and especially other developers and users
  • Ensure agreement on the work being done (it's too stupid to do a lot of work and only find when it's finished that it wasn't the right way to do it and it has to be all redone again)
  • Allow us to have product roadmaps
  • Provide the ability to help someone if he's lagging behind or has issues

A peek into XWiki SAS

  • Vincent works for the XWiki SAS company as its CTO but also as the person responsible for the Product Development Team of XWiki SAS, working on several products, including the XWiki open source product.
  • If you're curious, you can check the relationship between XWiki SAS & xwiki.org.
  • XWiki SAS has an internal roadmap process which comes up with a list of roadmap items that XWiki SAS think are interesting for the XWiki open source product to work on and that it's willing to sponsor by paying developers to work on them.
  • Vincent discusses with the XWiki SAS Product team members to assign roadmap items to XWiki SAS developers.
  • Vincent then sends the roadmap proposal mail on the devs mailing list, listing the items discussed by XWiki SAS with the defined assignees from XWiki SAS.

Development Flow

When you work on some XWiki code here's the typical workflow depending on what you're working on:

  • If you're working on Java classes, you develop in your IDE and unit tests there. Then you build your module with Maven and this generates some JAR file in target/. Then to test the functionality you write some functional tests. Running the functional tests generates a target/xwiki instance containing an XWiki instance that you can start manually if you wish to perform some additional manual tests. If you don't have any functional tests, you can use a standard XWiki standalone instance and copy the generated JAR file to that it replaces the original one in WEB-INF/lib (if you're changing an installed Extension, you need to replace the JAR in the /data/extension/repository folder).
  • If you're working on vm/js/css files, then you create/modify them directly in your XWiki standalone instance in the template/, resource/ or skin/ folders and just refresh the page using them (you may also need to clear browser cache for js/css). When it works you copy paste your modifications in the source tree. Note that you can also write functional tests as for the Java use case above.
  • If you're working on wiki pages, you also modify them directly in your running XWiki instance, and when you are happy with the result you export the pages as a XAR which you then unzip in the source tree (and you run mvn xar:format on them, see XAR plugin).

Automated Builds

XWiki has an automated build system and developers are asked to use it on their local machines to prove that their changes work before committing them to the Source Repository. See the Building page for more details.

Continuous Integration

See the Continuous Build page.

Special Project Roles

Committer Role

See Committership

Release Manager Role

Role definition:

  • In charge of performing a Release of XWiki (usually XWiki Commons/Rendering/Platform).
  • Is responsible to ensure that the dates defined in the Roadmap are achieved
  • Needs to coordinate with developers so that they are ready on the release day, this means warning several days ahead and shepherding them to finish their stuff on time or push stuff to the next release
  • Needs to follow the Release Plan Process. This app creates a release plan for releasing a version, listing all steps to be executed. It also keeps track of past Release Managers (and future ones) and of all past releases.

Process:

  • We have a list of declared Release Managers. Any committer willing to help the project is free to ask to perform a release!
  • We take turns, following the order (from top to bottom). When a newcomer is added he's inserted at the bottom of the list. 
  • If a committer cannot do the release for a reason or another (sick, holiday, workload, etc) he’s still responsible for the release and he/she needs to find a replacement. He also has the option to propose changing the release date but that needs to be accepted by the project. If he/she finds a replacement, it’s up to them to decide together who will do the release the next time that replacement’s time comes. 
  • All releases are included in this process: milestones, RCs, final, bugfix releases.
  • We switch Release Manager at each release (be it a milestone, RC, final or bugfix release). 
  • Once a Release is done, the Release Manager updates the Release Manager order by moving himself/herself at the bottom of the pile.

Build Manager Role

This is not a role we have all the time. We do it from time to time when our build quality starts to deteriorate and we want to get it back on track

Role definition:

  • Every week we have a different Build Manager chosen amongst the Committers
  • The Build Manager has the responsibility to get the build fixed ASAP whenever it's failing. His priority #1 during the week becomes monitoring and having the Build in working order (i.e. no failure)
  • By "Build" we mean the CI Build on https://ci.xwiki.org and by "failing" we mean anything that makes the build fail: tests, compilation, backward-compatibility checks, etc.
  • In order to fix build issues the Build Manager has several possibilities:
    • find out who caused the build to break and ask that person to fix it. That person cannot refuse that and must consider it his/her priority to fix it (or rollback the change that caused the build to fail)
    • rollback the issue that caused the build to fail
    • fix it himself/herself
    • find someone knowledgable in the failing domain and get him/her to fix the build.
  • The #2 priority of the Build Manager is to improve the XWiki build. Examples:
    • Fix more flickering tests
    • Improve the PO objects where needed, make sure that tests don’t use getDriver() (that should be only in POs).
    • Convert old func tests to docker-based tests
    • Add missing tests
    • Review/Analyze/Fix CI environment issues
    • Upgrade Jenkins and Jenkins plugins
  • There's a Build Manager Roster to log past Build Managers (and possibly future ones if some have expressed the wish to be the Build Manager for a specific week).
  • At the end of the Week the Build Manager must find the next Build Manager and hand over his duty. If he doesn't find anyone else, he remains the Build Manager. Note that for fairness the oldest Build Managers on the Roster need to accept to be the next Build Manager.
  • All committers must perform this duty and take turns

Roadmap Manager Role

Role definition:

  • Proposes new roadmap and ensure that there's always a roadmap ready before work is started for new XWiki versions

Process:

  • Once the RC version has been released and before the final version is released, send a Forum post proposal (tag: proposal, subjet: Roadmap for XS X.Y.Z)
  • Do this by discussing with the various commmitters to gather what everyone is interested in implementing for the upcoming release
  • Review leftovers from previous releases and consider them as candidates for the new release
  • Also update the Roadmap page with the proposal
  • Ask everyone with tasks in this roadmap to create JIRA issues and to mention them on the Roadmap page (using the jira macro)

Current manager:

Security Manager Role

Role definition:

  • Ensures that the XWiki project has a security policy and maintains it/improves it
  • Ensures that the security policy is enforced, e.g. make sure that confidential JIRA issues as categorized, and push for "high" severity ones (i.e. blocker ones) are put on the roadmap
  • Globally making sure that the XWiki project progresses on the topic of security.

Process:

Current manager:

Infrastructure Manager Role

Role definition:

  • Ensures that the servers used to develop XWiki are using latest versions and working fast and properly.

Process:

  • Review regularly the servers for problems and required upgrades, ideally once a month.

Current managers:

  • Weblate (l10n.xwiki.org): surli
  • xwiki.org and myxwiki.org: tmortagne
  • ci.xwiki.org: all committers
  • lists.xwiki.org: all committers
  • nexus.xwiki.org: all committers
  • forum.xwiki.org: vmassol

Versioning & Release Practices

Check the Versioning and Release Practices page.

Copyright header in source files

All files (including configuration files) must have the following copyright statement.

For Java & Javascript files

/*
 * See the NOTICE file distributed with this work for additional
 * information regarding copyright ownership.
 *
 * This is free software; you can redistribute it and/or modify it
 * under the terms of the GNU Lesser General Public License as
 * published by the Free Software Foundation; either version 2.1 of
 * the License, or (at your option) any later version.
 *
 * This software is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
 * Lesser General Public License for more details.
 *
 * You should have received a copy of the GNU Lesser General Public
 * License along with this software; if not, write to the Free
 * Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
 * 02110-1301 USA, or see the FSF site: http://www.fsf.org.
 */

For XML & Vue files

<!--
 * See the NOTICE file distributed with this work for additional
 * information regarding copyright ownership.
 *
 * This is free software; you can redistribute it and/or modify it
 * under the terms of the GNU Lesser General Public License as
 * published by the Free Software Foundation; either version 2.1 of
 * the License, or (at your option) any later version.
 *
 * This software is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
 * Lesser General Public License for more details.
 *
 * You should have received a copy of the GNU Lesser General Public
 * License along with this software; if not, write to the Free
 * Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
 * 02110-1301 USA, or see the FSF site: http://www.fsf.org.
-->

For Shell scripts, Properties & YAML files

# ---------------------------------------------------------------------------
# See the NOTICE file distributed with this work for additional
# information regarding copyright ownership.
#
# This is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as
# published by the Free Software Foundation; either version 2.1 of
# the License, or (at your option) any later version.
#
# This software is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this software; if not, write to the Free
# Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
# 02110-1301 USA, or see the FSF site: http://www.fsf.org.
# ---------------------------------------------------------------------------

For Bat files

REM -------------------------------------------------------------------------
REM See the NOTICE file distributed with this work for additional
REM information regarding copyright ownership.
REM
REM This is free software; you can redistribute it and/or modify it
REM under the terms of the GNU Lesser General Public License as
REM published by the Free Software Foundation; either version 2.1 of
REM the License, or (at your option) any later version.
REM
REM This software is distributed in the hope that it will be useful,
REM but WITHOUT ANY WARRANTY; without even the implied warranty of
REM MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
REM Lesser General Public License for more details.
REM
REM You should have received a copy of the GNU Lesser General Public
REM License along with this software; if not, write to the Free
REM Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
REM 02110-1301 USA, or see the FSF site: http://www.fsf.org.
REM -------------------------------------------------------------------------

For Velocity files

## ---------------------------------------------------------------------------
## See the NOTICE file distributed with this work for additional
## information regarding copyright ownership.
##
## This is free software; you can redistribute it and/or modify it
## under the terms of the GNU Lesser General Public License as
## published by the Free Software Foundation; either version 2.1 of
## the License, or (at your option) any later version.
##
## This software is distributed in the hope that it will be useful,
## but WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
## Lesser General Public License for more details.
##
## You should have received a copy of the GNU Lesser General Public
## License along with this software; if not, write to the Free
## Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
## 02110-1301 USA, or see the FSF site: http://www.fsf.org.
## ---------------------------------------------------------------------------

Coding Conventions

See the Code Style page.

File encoding

The following rules must be taken into account when the need for writing non-ASCII content in the XWiki sources arises:

  • All Java source files must contain only ASCII chars, Unicode escapes inside strings when needed, and XML entities in javadocs. Since we don't use @author tags, this should not be a problem.
  • All translation files must contain only ASCII chars and Unicode escapes (stronger than the Java properties specification).
  • All wiki documents sources must be stored in UTF-8.
  • Other XML files should always specify their encoding in the <?xml> header, and it should be UTF-8 as often as possible.
  • All other textual resources must be stored in UTF-8, minimizing the use of non-ASCII chars.

Currently, only the XML rules are broken, but this is not a serious problem since the XML reader can detect and use the charset/encoding specified in the XML header.

JIRA Projects

Creating a JIRA project

When creating a new Contrib JIRA project on https://jira.xwiki.org make sure to use the XWiki Contrib template (source code here):

jira-contrib-template.png

In addition perform the following steps:

  • Put as Project Lead the person requesting the project
  • Make sure to fill the URL part. Ideally it should point to documentation on extensions.xwiki.org. Failing that it should point to the GitHub project's page.
  • Create the appropriate versions. If the project has already had releases, recreate the versions and release them in JIRA with the correct release dates

For information this is what this template automates:

  • Creating a "JIRA Classic" project type
  • Selecting the proper "Category". For Contrib projects, it's "XWiki Contributed Projects".
  • Setting Workflows to use the "XWiki Workflow Scheme"
  • Setting Screens to use the "Basic Issue Creation Scheme"
  • Setting Fields to use the "XWiki Open Field Configuration Scheme"
  • Making sure that the default assignee is "Unassigned" and that it's not the Project Lead by default
  • Setting Permissions to use the "XWiki Open" scheme
  • Setting Issue Security to use the "XWiki" scheme
  • Setting Notifications to use the "XWiki Notification Scheme"

Retiring a JIRA project

Eventually, some XWiki projects will reach their end-of-life. This is often a good thing as new approaches or technologies have developed which reduce the need for the project. When a project retires, it is best to clean up loose ends to make sure outstanding users still have access to the code and the valuable project history is captured for future researchers.

Retiring steps:

  • Make the project 'Read-Only' by using the "XWiki Retired Project Scheme" permission scheme.
  • Change the name of the project by prepending it with the {RETIRED} label (e.g. Forum Application -> {RETIRED} Forum Application).
  • Move the project to the "XWiki Retired Projects" category. Edit the project and set its new category.
  • Update the project's description accordingly, informing the users that the project is no longer active.
  • If there are open issues that are still valid (i.e. they still need to be implemented), consider moving them to a new active JIRA project.

JIRA Best Practices

Here are some rules on how to use JIRA for XWiki projects.

Rule: Always put a JIRA issue reference in commit messages

The rationale behind this:

  • One consistent way to manage all the work to be done on the XWiki project. It also means there's no unaccounted work, meaning anyone can go to JIRA and query it and see what everyone has been working on.
  • Automated release notes/change logs. When we release a version we can simply do a JIRA extract and it'll give the full change log of what happened. This is really important for our users to see what has been done when in XWiki.
  • Traceability. When you use the JIRA issue number in your commit, the JIRA DVCS Connector Plugin can show the modified files directly from JIRA. This is quite useful later on, when someone is looking at a JIRA issue and wants to see what was modified. In addition Fisheye is also integrated with JIRA and when you browse the source repository you can see the JIRA issue associated with files.

Of course this shouldn't be done for any trivial things like adding a small javadoc, renaming a single variable, cosmetic changes, ignore files, etc.

JIRA issues can be marked "confidential", e.g. to hide the details of a security issue until it is fixed. Such issues should also be referenced in the commit message. However, avoid exposing the details of the issue in the commit message (some time will pass until the code is released) by choosing a more neutral description.

The general strategy goes like this:

  • When you plan to work on something, create a JIRA issue and assign it to yourself or simply assign an existing JIRA issue to yourself if one already exists
  • Implement it
  • Commit it with the JIRA issue number in the commit message. The format we follow is:
    <JIRA ID, e.g. XWIKI-1000>: <JIRA issue description>
    * <details>
    ...
    * <details>
  • Close the JIRA issue
  • Another acceptable variation is:
    • Implement something
    • When you want to commit it you realize you don't have any JIRA issue to put in the commit message so don't commit.
    • Create the JIRA issue
    • Commit with the JIRA number in the commit message

Rule: Don't create unnecessary issues

  • If you want to know whether you should create a JIRA issue or not, ask yourself the question: "is my change going to affect any user or any extension developer in any way"? If the answer is yes then you must create a JIRA issue
  • If you're fixing something related to build or some refactoring that doesn't impact users of XWiki directly then it's not mandatory to create an issue. However if you do, make sure you use the special component usually labeled "Development Issues only" in our JIRA projects. We're excluding issues in this category from our release notes.
  • Note that if you're upgrading a third-party dependency used by XWiki at runtime then you must create a JIRA issue using the special component usually labeled "Dependency Upgrades" and this needs to make it in the Release notes since it impacts XWiki users. If you're upgrading a third-party dependency only used at build time (for example the Selenium dependency) then you should use the "Development Issues only" component.
  • If you're fixing or reporting an issue related to code that's been introduced in the current version being developed, you shouldn't create a JIRA issue. Actually, if you did, you'd be hard-pressed to find the correct "Affects version" to use! emoticon_smile. Instead, use (or reopen) the existing JIRA that affects the current version being developed. If you're fixing it yourself, just use the same issue key. If you're reporting a problem, just reopen the issue and add a comment for the developer to see and fix before releasing.
  • If you're adding some translations on l10n.xwiki.org it's not necessary to create an issue in the l10n Project's JIRA. It would cause too much overhead.

Rule: Don't reopen issues closed for a released version

If you spot an incorrectly or incompletely fixed issue that was closed but the version it was closed for is already released (i.e. no more development can be done for that version), then you must never reopen such an issue. Instead, you should create a new issue that describes the work left to be done or the fact that the problem still exists after the attempted fix and link the new issue to the old one.

The main reason is that a fix for an issue must never span commits across multiple versions and we need to be able to clearly indicate what is the version where the issue was fixed in.

The only case where reopening such an issue is allowed is when the closed issue has no commits associated to it (i.e. closed as Won't Fix, Duplicate, etc.).

Rule: Use nice user-friendly titles

When you create a JIRA issue always take the time to put a nice title for the issue. The title must be understandable by a user so don't describe the issue technically but rather in what way it affects users. For example don't say "Fix the addXXX API to return an Array List" but rather "Allow creating several pages at once".

Rule: Don't set a "Fix Version" field for some issues

Issues that are closed with "Won't fix", "Duplicate", "Cannot reproduce", or "Incomplete" must have a "Fix Version" field set to "None" so that they don't appear in the Release Notes, as JIRA doesn't make the distinction visible and this causes confusion.

Note that “Solved by” issues must have a “Fix Version” set so that users can easily query JIRA to find in which version an issue has been fixed (rather than have to look in linked issues). This is useful for example for security issues which are closed as “Solved by” when they are fixed by other non-security issues. It’s also simpler for users to understand what’s been fixed in a given release, since the “Solved by” issue is usually described from a user perspective while the fixing issue is often more technical and more indirectly related.

Rule: Use the appropriate resolution type

The description of the different resolution types can be found directly in JIRA help.

Rule: Close issues that XWiki committers don't plan to implement

  • We close issues that we know we won't fix (using a "Fix For" value of "Duplicate", "Incomplete", "Cannot Reproduce" or "Won't Fix"). We might not want to fix them for several reasons but one reason is that the issue is for an old XWiki project version.
  • Leaving these issues open is not a good idea since:
    • It sends the wrong signals that we're going to fix the issue
    • The issue can still be found with a search even if closed
    • When we close it then the reporter can explain why it's so important for him/her, or not. Which we wouldn't know if we hadn't closed it
    • Contributors can reopen issues and attach a patch or just create a new issue
    • It means that for all other issues we plan to fix them at some point

Rule: Assign contributor when there's a Pull Request

  • If a contributor opens a PR, assign the corresponding JIRA issue to the contributor
  • If the contributor doesn't have a JIRA account then ask the contributor to create an account to assign him/her and in the mean time assign the committer merging the PR (so that if no account is ever created the committer stays assigned)
  • If you have to make substantial modifications to the PR, then you decide who gets assigneed (contributor or you), on a case by case basis.

Rule: create lowercase labels and without versions

The idea is consistency:

  • We use lowercase labels
  • Don't put specific versions in labels, instead use the Environment field to add more details about versions.
    • For example don't create a tomcat9 label, instead use the tomcat one and put "Tomcat 9.x" in the Environment field.

Rule: Always set an assignee when closing an issue

All closed issues must have an assignee set (even if the issue is closed as "won't fix", "duplicate", "invalid", etc). The assignee represents the person who's taken the call and became responsible for the issue closing.

Rule: Choose the right issue type

The definition of issue types are:

  • Bug: A problem which impairs or prevents the functions of the product.
  • Idea: An idea or wish (something not sure that'll happen but that would be cool)
  • Improvement: An improvement or enhancement to an existing feature or task.
  • New feature: A new feature of the product, which has yet to be developed.
  • Security: Security bug with confidential level
  • Task: A task that needs to be done.
  • Subtask: Must not be used. We need to remove it but it's hard to do now as lots of issues were created a subtasks in the past and we need to clean that up before we can remove this issue type.

Rule: Regression Handling

Regressions must be indicated in JIRA issues as follows:

  • The regression label must be set
  • The "Priority" field must be set to Blocker
  • The "Fix Version" field must be set with target versions being the next versions for the supported branches
  • A developer must be assigned ASAP to the issue.

Rule: Use the right labels

Use the following labels:

  • Usability labels
  • Security label
  • Regression label
  • BugFixingDay label
  • Test wanted label
  • blocker label for Blocker jira issues
  • responsive label for issues related to Responsive UI (especially XWiki UI on small screens, like mobile)
    • TODO: Decide what we do with mobile, ie whether we keep both labels or not
  • flickering label for issues related to flickering tests
  • performance label for issues related to performance (memory needs, speed of execution, scalability, etc)
  • Browser-related labels: ie, chrome, firefox, etc
  • Servlet-container-related labels: tomcat, jetty, etc
  • Database-related labels: mysql, mariadb, postgresql, oracle, etc
  • OS-related labels: windows, mac, etc
  • (Deprecated) configurationtesting label was used for issues related to Environment test (docker-based tests)
  • nestedspaces label for issues related to Nested Spaces or Nested Pages
  • onboarding label for simple issues that can be tackled by developers doing onboarding on the XWiki project

See also the use of lowercase labels.

Documentation Best Practices

The strategy is that when closing an issue in JIRA we make sure that there's documentation for it on xwiki.org (reference documentation) and on the Release Notes for the version corresponding to the JIRA Fix Version(s) field. This is done by filling the 2 corresponding fields in JIRA on the issue that is being closed. This allows several things:

  • Have up to date documentation on xwiki.org
  • Build Release Notes progressively so that when we perform the Release we don't have to wait a few more days to have everyone try to remember all the stuff they've done for the release...
  • Sometimes developers go on holidays (yep that happens emoticon_wink) and if they don't document the jira issues just after they're done, they forget to document them and it blocks the release or forces the Release Managers and other devs to document the jiras for them, which is painful and hard to do.

If an issue is a bug fix or contains changes that do not impact users or developers, then the string N/A should be used in the corresponding documentation fields in JIRA. This allows doing JQL queries and finding issues missing documentation.

Using bird names for Skins

We have decided to name XWiki official skins using bird names. We are currently renaming the existing skins. Any new skin must use a bird name. It's recommended to send a proposal to the list whenever a new skin has to be named.

Here are some potential ideas for future names:

  • Dove
  • Emu
  • Kiwi
  • Grasswren
  • Condor
  • Warbler
  • Kestrel
  • Redtail
  • Pigeon
  • Booby
  • Sparrow
  • Blackbird
  • Parrot
  • Peacock
  • Finch
  • Wren
  • Crow
  • Turkey
  • Thunderbird
  • Piasa (a local phenomenon, from a primitive painting on the cliffs on the eastern shore of the Mississippi, attributed to the Piasa tribe, which looks rather like a griffin)
  • Thunder Chicken
  • Kingfisher
  • Swallow
  • Hummingbird

Back-end Development Practices

Component Development

Read the Component Module documentation for details on how to write components.

Backward Compatibility

We pay a lot of attention to backward compatibility. This is why we're using the Revapi Maven plugin in our builds to ensure we don't break public APIs. 

Specifically we check for the following type of backward compatibility issues (See full list provided by Revapi):

  • Binary incompatibilities
  • Semantic incompatibilities

Note that we don't check for source incompatibilities since it's too strict and we want to be able to change code (like add missing generics to return types) without breaking the build and having to put ignores in the Revapi configuration of the build.

Since it takes time to stabilize an API we've introduced an annotation named @Unstable (see below).

When you need to add a new method to an interface there are 2 solutions to preserve backward compatibility:

  • Create a new interface with the new method and deprecate the old one. This means that code using the old interface must be modified to support the 2 interfaces, and that's not easy.
  • (Recommended) Use Default Methods (introduced in Java 8).
    • Note that the default method should not throw an exception in general (like throw new UnsupportedOperationException("Not supported");), since that would mean that the code calling the default method will fail. As such, it cannot be considered as backward compatibility.
    • However there's an exception: if the new method throws a checked exception in its signature then it's fine to throw it in the default method's implementation. Similarly, if the new method is not throwing a checked exception but it's documented to throw some runtime exeption that is supposed to be handled in the call chain, then it's also fine.

Deprecating Code

We follow 2 steps when it comes to deprecating code:

  • First step: We start by deprecating it using the @deprecate and @Deprecated Javadoc and java annotations. The code remains where it was before the deprecation happens.
  • Second step: When our own code doesn't use any of the deprecated APIs anymore then move the code to a legacy module. This can be done as soon as the deprecation is added.

This has the following advantages:

  • Our code remains clean of old deprecated APIs
  • Deprecated code is cleanly separated from new ways of doing things
  • When new users download the our code they see the new ways (same for javadoc generation)
  • Users wanting to use the old APIs can still do so
  • We never remove APIs from the legacy modules by default. However if we really need to do so (for some technical reason for example), we do it on a case by case basis with a VOTE.

In the future, our distribution may not package legacy JARs by default and users wanting to use older APIs may need to install the legacy modules. This is an effort to move users to the newest API as soon as possible while still allowing them to use the old API it they're not ready to move yet (but at least they'll be aware of the changes and about what they have to fix). However in order to provide a good user experience, before doing this we need to add support for Legacy jars in the Extension Manager.

Revapi Ignores

At build time, Revapi tells us when we break backward compatibility, by marking the Maven build in error. We then need to review the error and decide if it's legitimate or not. If we decide it's legitimate, we then add an ignore in a pom.xml in each repository (commons, rendering and platform. To know the location search for <revapi.differences>).

There are 4 types of cases:

  • Real breakages that we still want to do but the users need to be warned since existing code using these APIs will or may fail at runtime.
    • Mark the ignore with <criticality>highlight</criticality>, e.g.
      <item>
        <code>java.method.removed</code>
        <old>method void org.xwiki.diff.xml.XMLDiff::xxx()</old>
        <justification>Not really usable API added by mistake.</justification>
        <criticality>highlight</criticality>
      </item>
  • Real breakages but happening on @Unstable code.
    • Mark the ignore with <criticality>documented</criticality>
  • Semantic changes that are potentially breaking in severity(like adding an annotation) but that, after analyzing we consider to not be a breakage.
    • Mark the ignore with <criticality>allowed</criticality>
  • Revapi bugs or limitations, that are not breakages in our opinion (could also be APIs moved into another Maven module)
    • Mark the ignore with <criticality>allowed</criticality>

Note that the criticality information is then used when producing our release notes (e.g. we don't list allowed items in the Release Notes).

Deprecation Rules

For details and rationale about those rules below see this thread.

  • Rule: Location of Legacy modules
    • Each Git repository needing legacy modules provides a *-legacy module for holding legacy modules. For example:
      • For Commons, xwiki-commons-core/xwiki-commons-legacy/
      • For Platform, xwiki-platform-core/xwiki-platform-legacy/
      • For Rendering, xwiki-rendering-legacy/
  • Rule: Use AspectJ to move deprecated APIs to Legacy modules
  • Rule: Legacy modules replace modules from where they come from
    • Each Legacy module replace the non-legacy module it corresponds to. This means that the user must have only 1 JAR for a given module: either its legacy version or it's non-legacy version but should never have both.
  • Rule: Annotate code with version to show deprecations
    • Use both @Deprecated annotation and the @deprecated javadoc tag and specify the version when the deprecation was added. For example:
      /**
       * @param time the time in milliseconds
       * @return the time delta in milliseconds between the current date and the time passed as parameter
       * @deprecated replaced by {@link com.xpn.xwiki.api.Util#getTimeDelta(long)} since 1.3M2
       */

      @Deprecated
      public int XWiki.getTimeDelta(long time)
      {
         return this.util.getTimeDelta(time);
      }

@Unstable Annotation

This annotation can (and should) be used by developers whenever new public API is introduced (in addition to also adding a @since annotation). This annotation means that the code is subject to change at any time (in which case it'll appear in the backward-compatibility reports but with the explanation that it's an Unstable API). From a user point of view, it means they should use classes or methods annotated with the @Unstable annotation with caution. It means they need to be prepared to modify their code. Alternatively they can decide to not use classes/methods annotated with @Unstable and wait for them to come out of unstability.

In order to prevent code to remain annotated with @Unstable forever, we've defined some rule;

  • Any code annotated with @Unstable can only remain in this state for a maximum of 1 full cycle. For example if a unstable API is introduced in 4.4, it'll have to come out of unstability before 6.0M1 is released (thus having 1 full cycle, i.e. the 5.x cycle). This means that the worse that can happen is for an unstable API added in N.1 which will have to be removed before N+2 Milestone 1 is released.
  • However, the previous rule is only a maximum and developers are encouraged to remove the unstable annotation whenever they feel that the API should come out of unstability. When this happens the standard deprecation mechanism then kicks in.

The following automated checks related to the @Unstable annotation are performed in the build:

Configuration Property Naming

XWiki uses 2 configuration files at the moment:

  • An old one, named xwiki.cfg for old core code and which eventually will go away when all code will have been rewritten using component and extracted as separate modules
  • A more recent one that must be used for new configuration properties required by new code, named xwiki.properties

The naming rule for xwiki.properties is:

  • Use <module>.<propertyName>. Ex: rendering.linkLabelFormat
  • For submodules use <module>.<submodule>.<propertyName>. Ex: rendering.macro.velocity.filter
    • Note: Unfortunately you'll find some properties not following this rule in xwiki.properties. This is because we've been bad in following this rule and doing code review to ensure it was followed. It was also voted on the mailing list but not documented here in the past.
  • Use camelcase for the property name itself. ex: linkLabelFormat

Translation Best Practices

General

  • The XWiki Core committers maintain one version, the English one and more precisely the en_US one. This means:
    • The ApplicationResources.properties file (without any language suffix) represents the en_US version
    • We should use "customize" instead of "customise" or "color" instead of "colour" emoticon_wink
  • Other translations are maintained by the community at large on l10n.

Translation Property Naming

When content requires localization you should use the following rules:

  • If the content is inside a wiki page, then create a wiki page named Translations (or *Translations if there's a need to have several translation pages) in the space of the application containing the content to translate. This page must be registered using a XWiki.TranslationDocumentClass xobject.
  • If the content is inside some .vm files or in Java, then create an ApplicationResources.properties file in the src/main/resources directory of the module using the translation (so that it's packaged at the root of the JAR).
  • Generally speaking the translations must be part of the extension containing the content to translate.
  • New translation resources must be added on http://l10n.xwiki.org and also in the Weblate synchronization scripts.
  • Special case for extensions containing only wiki Macros which go in the Macros space: the Translations file should be located in the same space as the macro.
  • Follow the Conventions for naming the translation keys

Translation Property Deprecations

  • When deleting a key, it should be moved to the deprecated section at the end of the file, linked to the first version in which it started to be deprecated. If there's no such section in your translation file you must add one, using the following syntax:
    #@deprecatedstart
    ...
    ##################################################
    ## until <version that deprecated the keys below>
    ##################################################
    ...

Not supported by the new Weblate based l10n platform.

  • When renaming a key, in addition to move the deprecated key to the deprecation section, you should also add a #@deprecated comment pointing to the new key, using the following syntax:

Example:

###############################################################################
## Deprecated
## Note: each element should be removed when the last branch using it is no longer supported
###############################################################################

## Used to indicate where deprecated keys start
#@deprecatedstart

#######################################
## until 10.1
#######################################

job.log.label.refactoring/rename=Rename log
job.log.label.refactoring/copyAs=Copy log

...

#######################################
## until 3.5
#######################################

#@deprecated platform.livetable.results
xe.livetable.results=Livetable Results
...

Moving Translations

In general translations should not be moved as this could cause backward compatibility problems with existing extensions, and thus it's better to deprecate and create new keys. However it's a good idea to copy all existing translations to the new key whenever possible. This should be done on the source files and committed in Git (it's automatically synced on l10n.xwiki.org with a commit hook). A script is available to help performing such migration: https://github.com/xwiki/xwiki-dev-tools/blob/master/weblate-scripts/migrate_keys.py.

Migrating away from the Old Core

Starting in 2006 we've started to work on splitting the code that's currently located in the xwiki-platform-oldcore module into various modules, each one specific to a given domain. Before 2006 the whole of XWiki was located in that oldcore. The work isn't over and the current strategy is the following:

  • Continue extracting code from oldcore and put it in its own domain modules
  • We allow new modules to depend on oldcore provided that oldcore itself doesn't depend on these new modules obviously (as otherwise it would create a cyclic dependency). When that happen it usually means we need to move the code in oldcore that uses thee new module outside of oldcore in an existing module or in a new one.
  • At some point all that should remain in oldcore is the old Model itself.
  • We need to continue the work we started about writing the New Model and put it in some new module, at which point we'll need to migrate our code to use the new Model progressively and then the last step will be to remove oldcore altogether.

It's important that we stop adding new stuff to oldcore and instead always try to extract stuff outside of it as otherwise this means accruing even more our technical debt and making it harder to completely remove oldcore.

User Interface Extension Point Naming

User Interface Extension Point (UIXP) must follow the following pattern:

<groupId>.<moduleName>.<uixpQualifier> where:

  • <groupId>: reuse maven’s groupId in which the UIXP is declared. Lower case, dot separated.
  • <moduleName>: reuse maven’s module in which the UIXP is declared. Lower case, dot separated.
  • <uixpQualifier>: descriptive name of the UIXP. Single camel case identifier.

Example: org.xwiki.platform.user.profile.menu

The User Interface Extensions (UIXs) contributing to an UIXP must follow the following pattern:

<groupId>.<moduleName>.<uixpQualifier>(.<uixQualifier>) where:

  • <groupId>: reuse maven’s groupId in which the UIX is declared. Lower case, dot separated.
  • <moduleName>: reuse maven’s module in which the UIX is declared. Lower case, dot separated.
  • <uixpQualifier>: reuse the uixpQualifier of the UIXP where the UIX contributes.
  • <uixQualifier>: optional, used only in case of ambiguity, for instance if the UIX is declared in the same module as the UIXP, or if several UIX are declared in the same module. Single camel case identifier.

Examples of UIX contributing to the org.xwiki.platform.user.profile.menu UIXP:
org.xwiki.platform.notifications.menu
org.xwiki.platform.user.profile.menu.userMembership
org.xwiki.platform.user.profile.menu.userNetwork

Front-end Development Practices

HTML & CSS Best Practices

See our HTML & CSS code style.

Icons

When using icons in content, they must rely on the XWiki icon set.

When adding new icons to the XWiki icon set, their names must follow our icon naming conventions.

JavaScript Best Practices

  • JavaScript code should be organized in modules following the Asynchronous module definition (AMD) specification. At the moment we're using RequireJS for this:
    require(['jquery', 'xwiki-suggestPages', 'xwiki-events-bridge'], function($) {
      ...
    });
  • XWiki JavaScript modules should have their name prefixed with 'xwiki-' in order to avoid conflicts with external modules. E.g.:
    define('xwiki-diff', ['jquery', 'xwiki-events-bridge'], function($) {
      ...
    });
  • The recommeded places to store and load JavaScript modules (dependencies) from are:
  • Integrating external JavaScript libraries should be done only through WebJars.
  • Avoid using inline script tags or inline JavaScript through HTML attributes (like onclick). Doing this can cause problems with event handlers and is sometimes WCAG invalid.
  • The recommended way to inject a JavaScript module into the current web page is:
    • either by using the JSX plugin (e.g. $xwiki.jsx.use(...)) from a server-side script (e.g. to load the JavaScript module that serves as your entry point)
    • or by declaring the module as a dependency of another JavaScript module
  • We recommend loading JSX using "On demand only". That is: load your JavaScript code only where / when it is needed. Avoid using "On this wiki".
  • Localizations in Javascript should use a dedicated module and the specific loader
  • Avoid mixing JavaScript with server-side scripting (e.g. Velocity), especially for JavaScript code packaged as WebJar, because in this case the scripts are evaluated long after the JavaScript code is minified and the minifier can rewrite the JavaScript code in a way that breaks the embedded server-side scripts. If this is not possible then use this pattern to separate the JavaScript-only code from the embedded scripts:
    /*!
    ## Velocity code here.
    #set ($paths = ...)
    #set ($l10n = ...)
    ...
    #[[*/

    // Start JavaScript-only code.
    (function(paths, l10n) {
     "use strict";

      require(...);

    // End JavaScript-only code.
    }).apply(']]#', $jsontool.serialize([$paths, $l10n]));
  • Check the list of supported browsers and MDN or websites like caniuse.com before using newer JavaScript APIs or syntax sugar. Note that the minifier might be able to rewrite your code in order to support older browsers, but you need to check.
  • When updating existing non-AMD code (e.g. when fixing bugs) try to rewrite the code (or parts of it) as AMD. Avoid in-line mixing of non-AMD code with AMD code because it makes the code harder to read and maintain.
  • Rewrite old code based on Prototype.js by using jQuery or plain JavaScript when possible
  • We recommend enabling the Strict mode
  • The JavaScript code should pass the following rules to ensure good quality and common code style:
    // jsHint rules, see https://github.com/jshint/jshint/blob/master/examples/.jshintrc
    {
     "camelcase": true,
     "maxparams": 5,
     "maxdepth": 3,
     "maxstatements": 20,
     "maxcomplexity": 10,
     "maxlen": 120
    }

    // ESLint rules, see https://eslint.org/docs/rules/
    {
     "camelcase": "error",
     "max-params": ["error", 5],
     "max-depth": ["error", 3],
     "max-statements": ["error", 20],
     "complexity": ["error", 10],
     "max-len": ["error", 120]
    }
  • JavaScript modules should have automated unit-like tests.

Check FrontendResources for more JavaScript-related documentation.

Backward compatibility and deprecation

As for back-end development, we should take care of backward compatibility. There are applications developed on top of XWiki that use their own JavaScript APIs. The rule for deprecating a JavaScript API (be it an object or a method of an object) is to move or write code to maintain compatibility in the compatibility.js file under the xwiki folder. We should always wrap deprecated code to log its usages. This will make the life of applications developers much easier.

Example of deprecation code:

/**
 * Deprecated since 1.9M2
 */

window.displayDocExtra = XWiki.displayDocExtra;
window.displayDocExtra = window.displayDocExtra.wrap(
 function(){
    warn("window.displayDocExtra is deprecated since XWiki 1.9M2. Use XWiki.displayDocExtra instead.");
   var args = $A(arguments), proceed = args.shift();
   return proceed.apply(window, args);
  }
);

XWiki Standard should always work fine when running without the compatibility file. Before deprecating a method or object, we should ensure it is not used anywhere any longer in XWiki Standard distribution.

Bug Reporting

The bug reporting system resides at https://jira.xwiki.org/

XWiki.org Rules

Some rules to follow when making modifications on xwiki.org.

Don't remove important pages when they are moved

Rationale: Users will save links and when they navigate to them later on these links will be broken.
Solution: Add a redirect script to redirect to the new page
Implementation: Use this script:

{{velocity}}
$response.sendRedirect($xwiki.getURL("newSpace.newPage"))
{{/velocity}}

Don't use URLs for links to *.xwiki.org

Rationale: If we change the URL format or the domain the links will be broken
Solution: Use the wiki:Space.Name notation

XWiki Days

See XWiki Days.

Application Development

See Application Development Best Practices.

Build Best Practices

  • For the general directory structure, see SourceRepository.
  • Directory names: We're currently using the a directory name corresponding to the Maven artifactId. For example: xwiki-platform-watchlist-ui for a pom.xml having <artifactId>xwiki-platform-watchlist-ui</artifactId>

    Note that this scheme is causing some issues on Windows since it generates long path names and Windows usually supports only 255 characters. It's possible that we'll change this in the future... or not, since you can use Cygwin on Windows to have longer path names.

  • Maven groupIds: org.xwiki.<short name of top level project>. For example org.xwiki.commons for XWiki Commons, org.xwiki.rendering for XWiki Rendering and org.xwiki.platform for XWiki Platform
  • Maven artifactIds: xwiki-<short name of top level project this module belongs to>-<module name>-<qualifiers>. For example:xwiki-platform-watchlist-ui. There are some conventions used:
    • -api for modules providing APIs
    • -ui for modules generating XARs
    • -test for functional test parent POM
    • -test-pageobjects for functional test Page Objects modules
    • -test-tests for modules containing and executing the functional tests (non-docker tests)
    • -test-docker for modules containing and executing the functional tests (docker-based tests)

Top Level Extensions

Rationale/Need:

  • Be able to extract some apps from xwiki-contrib that the XWiki Dev Team would like to maintain. Example: File Manager app developed by Marius when it’ll have had some releases and tests (if it doesn’t have some already!), GitHub Stats app used on xwiki.org, Meeting Manager App, Forum App, etc 
  • Be able to extract some extensions currently located in xwiki-platform but not released with XWiki Standard so that they can have a different release cycle (examples: FAQ app, IRCBot extension, JIRA macro, etc). Having different release cycle allow to release new versions quicker to our users (bug fixes, new features). 

Governance:

  • Extensions are VOTEd in on a case by case basis. 
  • Each voted extension has its own Git Repository in the “xwiki” organization (so that each extension can be released independently of each other). 
  • When moving an extension either from xwiki-contrib or from xwiki-platform, keep its Git history as much as possible or simply donate the repo to the xwiki organization. 
  • FTM extensions bundled by default with XWiki Standard still remain in XWiki Commons/Rendering/Platform. 
  • The Git repository name must be of the form xwiki-<short project name>. <short project name> must be part of the VOTE. 
  • All XWiki Development rules apply 
  • Each extension has a Release Manager defined and he’s responsible for defining its own Roadmap/Release notes (if need be), on the extension page on e.x.o and perform the releases or ensure the extension is released regularly when there are changes. 
  • Each extension must follow these criteria for being VOTEd:
    • A Release Manager needs to be defined in the proposal 
    • The extension must have had several releases already (i.e. someone wanting to propose a new extensions that doesn’t exist would start in xwiki-contrib for ex and prove that his extension works and is useful by doing several releases and creating the pages on e.x.o) 
    • It must follow our best practices (coding practices, tests, etc) and follow the apps best practices (for apps).
    • It must have one or several integration or functional tests (for apps) to prove that it works. This allows to prove the app continues working when XWiki progresses 
    • The main contributors of the extensions must agree about the move. If they have the “level" to be an xwiki dev committer then they should be voted in. If not then either they’re ok to send Pull Requests or the extension should not be moved. 
  • If an extension ceases to work or if its quality becomes too low, it can be moved to xwiki-contrib with a VOTE 
  • We create one JIRA project per extension 
  • We create a new JIRA Category called “XWiki Extensions” 
  • We put the extensions in our CI at http://ci.xwiki.org 
  • The Java package must follow the same rule as for XWiki Platform, i.e. org.xwiki.<short project name>. Exceptions would need to be discussed. 
  • The group id for extensions having their own repo must be org.xwiki.<short project name>. The <short project name> needs to be part of the VOTE when proposing a new extensions. 

Retiring a module

When a module needs to be moved to the XWiki Attic, the following process must be followed:

  • Send a VOTE propose on the Forum
  • Once agreed:
    • If you're retiring the whole GitHub repo, then simply move it on GitHub and if you're only retiring a part of the repo then Extract out the module.
    • Retire the JIRA project
    • Update the page on e.x.o to indicate this with a warning and information in the Compatibility section.

Extracting out a module

When a module is moved to XWiki Contrib or to the XWiki Attic, it needs to be done without loosing the git history. For example for extracting out xwiki-platform-blog to xwiki-contrib:

When moving to XWiki Contrib you'll need to do the following:

  • Change the groupId/artifactIds and directory names
  • Use the org.xwiki.contrib:parent-* as the parent, using the LTS version (e.g. 13.10 if 13.10.x is the "LTS branch")
  • Use the xwiki.extension.features EM property in the POMs to provide retro-compatibility
  • Continue using the same version to signify it's not a new extension. E.g. if the version in XWiki Platform was 14.7-SNAPSHOT when moving it out, keep using that. Then, once the extension has been released, it'll decide how often to release and how to increase the version (but the version must be increased from that number).
  • Update the extension id on the e.x.o page for the extension and mention the move there in the compatibility section.
  • More generally follow the best practices for a Contrib project (new JIRA project, changes to the top level POM, README file, etc).

Merging in a module

When a module is moved from XWiki Contrib to XWiki Platform, it needs to be done without loosing the git history. For example, for merging in application-ckeditor to xwiki-platform:

  • Call git subtree add --prefix xwiki-platform-core/xwiki-platform-ckeditor [email protected]:xwiki-contrib/application-ckeditor.git master where the prefix is the new location of the merged in extension in XWiki Platform hierarchy
  • Adapt the code to conform to XWiki Standard coding style, integrate the new module to the hierarchy, etc.
  • Push the branch

It is also important to test the build on an environment similar to the release machine to detect problems early (e.g., missing command line tools).

JMX Monitoring

All XWiki monitoring APIs must be implemented and exposed as JMX MBeans. See also how to Monitor an XWiki instance.

Best practices:

  • Always use XWiki's JMXBeanRegistration component to register a MBean.
  • Make sure you don't forget to unregister your MBean when needed. Usually registration is done in component's Initializable#initialize() methods and unregistration in Disposable#dispose() one.

Example:

public interface JMXVelocityEngineMBean
{
    TabularData getTemplates();
}

public class JMXVelocityEngine implements JMXVelocityEngineMBean
{
   /**
     * The Velocity Engine for which to return management data.
     */

   private VelocityEngine engine;

   /**
     * @param engine the Velocity Engine for which to return management data
     */

   public JMXVelocityEngine(VelocityEngine engine)
   {
       this.engine = engine;
   }

   /**
     * {@inheritDoc}
     * @see JMXVelocityEngineMBean#getTemplates()
     */

   public TabularData getTemplates()
   {
      ...
   }
}

@Inject
private JMXBeanRegistration jmxRegistration;
...
JMXVelocityEngineMBean mbean = new MyBean(engine);
   this.jmxRegistration.registerMBean(mbean, "type=Velocity,domain=Engines,name=" + key);

Add a new committer

See Committership.

Security

  • We follow the Security Policy. This documents contains both the policy and also development topis (such as how to handle security issues by developers, how to add a new member to the security sites, etc)
  • All developers must also follow the best practices regarding security.

Rendering Macros

Macros written for XWiki Platform should be written in Java (by opposition to wiki pages) for the reasons mentioned in the comparison table.

Tags:
   

Get Connected