Release Plan Help
- Initial Backward Compatibility Check
- Verify JIRA issue
- Log on Release machine
- Set up your identity
- Update Release scripts
- Configure the Java Version
- Update Translations
- Compute list of updated languages
- Build the release
- Generate Code Contributors list
- Handle Backward Compatibility Actions
- Clean up identity
- Blog post on xwiki.org
- Update Wikimatrix
- Forum Announcement
- Announce on Twitter
- Rebuild Debian Distribution
- Force Extensions Update
- Update Docker
Contains instructions to perform various release steps in the Release Plans.
Initial Backward Compatibility Check
This initial step checks if the previous release cycle, or more recent bugfix releases done since, have properly set the xwiki.compatibility.previous.version property in the top level POM of xwiki-commons (for the branch you are currently releasing).
For example, if the current release to be made is for 4.2-milestone-3 and, in the meanwhile, there have been released bugfix versions for 4.1 (4.1.4 being the latest released), you need to ensure that the value for xwiki.compatibility.previous.version is 4.1.4.
Verify JIRA issue
If there are opened JIRA issues for the version you are currently releasing (fixVersion is set to the current release), you have two options:
- The issue has no code committed yet (Commits tab shows nothing). In this case, you need to push the issue to the next development version. (e.g. fixVersion = 6.3-milestone-1 will be moved to fixVersion = 6.3-milestone-2)
- There is some code already committed (Commits tab is not empty). In this case, you need to contact the assigned developer of the issue and find out if the committed code is complete:
- If the code is complete, you can close the issue (or ask the dev to do it)
- If the code is not complete, you can close the issue and ask the dev to create a new issue for the remaining work to be done for the next version.
In all cases, all opened issues must be closed before the release can be done.
Log on Release machine
me@home:~$ ssh [email protected]
# Get on the agent machine
maven@maven:~$ a1-1
Set up your identity
The first time
- Configure your Github profile to add the SSH key of the agent so that the commits can be pushed to Github with your user (To get the agent SSH key, do cat ~/.ssh/id_rsa.pub). To verify it works, do the following on the agent machine: ssh -T [email protected]. It should reply with Hi <your user id>! You've successfully authenticated, but GitHub does not provide shell access..
- Create your own GPG key if you don't already have one: gpg --gen-key
- Publish your public GPG key, if you have not already done so: gpg --keyserver hkp://pool.sks-keyservers.net --send-key <keyID>
- Import your GPG key:
- On your local computer run: gpg --list-secret-keys
- Export your key in a secret.key file: gpg --export-secret-keys «keyID» > secret.key where keyID is the 8 characters long hexadecimal string on the sec line
- Copy the secret.key file to the agent machine (you may need to copy to maven.xwiki.org first and then to agent-1-1)
- From your local computer, upload the key to maven.xwiki.org: scp secret.key [email protected]:/home/maven
- From maven.xwiki.org (/home/maven): scp secret.key a1-1:/home/hudsonagent/<yourname>.gpgkey (give it a name)
- The key can be stored on the agent, since it's password protected, so that you don't have to upload it next time, and just re-import it.
- TODO: find out how we can keep the keys imported and tell maven which one to use during the release.
- Remove the key from maven.xwiki.org after you have uploaded it to agent-1-1: rm secret.key
- On the agent machine, run gpg --import <yourname>.gpgkey
- You need to have a .gitconfig.<yourusername> file. You can do that by copying one of the existing gitconfig file
- Make sure your gitconfig file has the key signingkey in the [user] section (the value is your key id which you can get with gpg --list-secret-keys).
- Copy your .gitconfig.<yourusername> file as the main gitconfig file: cp .gitconfig.<yourusername> .gitconfig
The other times
- Configure your Github profile to add the SSH key of the agent so that the commits can be pushed to Github with your user (To get the agent SSH key, do cat ~/.ssh/id_rsa.pub). To verify it works, do the following on the agent machine: ssh -T [email protected]. It should reply with Hi <your user id>! You've successfully authenticated, but GitHub does not provide shell access..
- Make sure previous release manager did not left his GPG key
gpg --delete-secret-and-public-keys `gpg --list-secret-keys | grep ^sec | cut -d/ -f2 | cut '-d ' -f1` - Import your GPG key, run gpg --import <yourname>.gpgkey
- If you did not leave your secret key file on the agent machine, you have to re-upload it (see The first time section above).
- Copy your .gitconfig.<yourusername> file as the main gitconfig file: cp .gitconfig.<yourusername> .gitconfig
Script to automatize this step
- Put this script on your local machine.
- Replace the USER_NAME variable.
- Make sure you have exported your secret key in a file called secret.key in your user directory.
- Don't forget to give execution right to the file.
- Execute and enjoy!
USER_NAME='gdelhumeau'
function log {
## Colors
RESET_TEXT_COLORS='\033[0m'
GREEN_TEXT='\033[1;32m'
BLACK_BACKGROUND='\033[40m'
echo -e "${GREEN_TEXT}${BLACK_BACKGROUND}* $1${RESET_TEXT_COLORS}"
}
log "Copy secret key to the maven machine"
scp ~/secret.key "[email protected]:/home/maven/${USER_NAME}.key"
log "Put it on the agent1-1"
ssh [email protected] "scp /home/maven/${USER_NAME}.key a1-1:/home/hudsonagent"
log "Delete it from maven.xwiki.org"
ssh [email protected] "rm /home/maven/${USER_NAME}.key"
log "Import the key in agent1-1"
ssh [email protected] "ssh a1-1 'gpg --import /home/hudsonagent/${USER_NAME}.key'"
log "Delete it from agent1-1"
ssh [email protected] "ssh a1-1 'rm /home/hudsonagent/${USER_NAME}.key'"
log "Copy gitconfig"
ssh [email protected] "ssh a1-1 'cp /home/hudsonagent/.gitconfig.${USER_NAME} /home/hudsonagent/.gitconfig'"
log "Display agent1-1 SSH key (to put on https://github.com/settings/keys)"
printf "\033[0;91m\033[0;100m"
ssh [email protected] "ssh a1-1 'cat /home/hudsonagent/.ssh/id_rsa.pub'"
log "Connect to hudsonagent"
ssh -tt [email protected] "ssh -tt a1-1"
Update Release scripts
hudsonagent@vagent-1-1-dev:~$ cd xwiki-dev-tools ; git pull --rebase ; cd ..
Configure the Java Version
Depending on which branch you are releasing from you may have to use a different Java version than what is used on master. To change the Java version you need to remove the "java" symbolic link from the home folder and recreate it to point to the "javaX" folder that corresponds to the Java version you have to use.
hudsonagent@vagent-1-1-dev:~$ ls -al | grep java
# Change the Java version
hudsonagent@vagent-1-1-dev:~$ rm java
hudsonagent@vagent-1-1-dev:~$ ln -s java7 java
# See if the change has been taken into account.
hudsonagent@vagent-1-1-dev:~$ java -version
Update Translations
For a release candidate, the translations were already merged (when the Pull Requests were accepted) into the master branch which you have just forked.
For a final version, there may be translations added since the RC that we need to integrate. Perform the following steps:
- Move to xwiki-trunks:hudsonagent@vagent-1-1-dev:~$ cd releases/xwiki-trunks/
- Run the update goal of the script with your branch (e.g. stable-10.5.x):hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks$ ~/xwiki-dev-tools/weblate-scripts/apply_translations.sh update stable-10.5.x
- Review the changes for anything amiss.
- You need to go through each project (xwiki-commons, xwiki-rendering, xwiki-platform)
- Do a git status to spot changes.
- Do a git diff --cached
and visually review modified translation keys and spot problems (like spam). Do not spend too much time here, since major issues should have been reviewed by the committer that merged the Pull Request, a quick scan is enough to spot obvious problems. - No need to do a git add at this point since it will be done for all projects below.
- Do a git diff --cached
- Push / commit the translation changes on the branch of the version you are releasing (e.g. stable-10.5.x):hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks$ ~/xwiki-dev-tools/weblate-scripts/apply_translations.sh push stable-10.5.x
Compute list of updated languages
Collect the list of updated languages to publish in the Release Notes:
- cd to each repository (xwiki-commons, xwiki-rendering, xwiki-platform)
- Run the list_translation_changes.sh script to list all relevant modified translation files since the previously released final version and generate the list of modified languages (notice the usage of the --diff parameter to help you decide if a translation language should be counted or not, see below for more details):## list_translation_changes.sh start_commit end_commit [options]
## Example when releasing a final version (e.g. 10.7):
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks/xwiki-commons$ ~/xwiki-dev-tools/weblate-scripts/list_translation_changes.sh xwiki-commons-10.6.1 stable-10.7.x --diff | more
## Example when releasing a RC version (e.g. 10.7RC1):
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks/xwiki-commons$ ~/xwiki-dev-tools/weblate-scripts/list_translation_changes.sh xwiki-commons-10.6.1 xwiki-commons-10.7-rc-1 --diff | more
## Other examples on some already released versions (in case the need arises):
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks/xwiki-rendering$ ~/xwiki-dev-tools/weblate-scripts/list_translation_changes.sh xwiki-rendering-10.3 xwiki-rendering-10.4 --diff | more
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks/xwiki-platform$ ~/xwiki-dev-tools/weblate-scripts/list_translation_changes.sh xwiki-platform-10.3 xwiki-platform-10.4 --diff | more - Merge the list of updated language codes across all the repositories (from the output of the commands above) and update the Release Notes in the Translations section with these language codes.
- Only count what should be counted:
- Skip duplicates and sort the final list alphabetically.
- Only count what should be counted:
Build the release
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks$ ~/maven-release.sh
Restart the release
If the build failed or if you wish to release again, perform the following steps:
- Set the version that you're building, e.g. hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks$ export VERSION='4.2-milestone-3'
- Navigate to the repositories that failed to build (e.g. xwiki-platform) and clean up. For example for xwiki-platform:hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks$ cd xwiki-platform
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks/xwiki-platform$ ~/restart-release.sh - Edit the maven-release.sh script and comment out the previously successfully released repositories so that you don't release them again.hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks/xwiki-platform$ cd ..
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks$ vim ~/maven-release.shGo to the end of the maven-release.sh file and scroll up, until you reach this section:
# Wrapper function that calls release_project for each XWiki project in order: commons, rendering, platform, enterprise, manager.
# This is the main function that is called when running the script.
function release_all() {
...Inside this method, comment out the previously successful top level projects and leave uncommented the top level projects that still need to be released, including the one we are resuming from. In this example, we are resuming from xwiki-platform but xwiki-commons and xwiki-rendering were already successfully released, so we are skipping them:
...
echo "*****************************"
echo -e "\033[1;32m Releasing xwiki-commons\033[0m"
echo "*****************************"
# release_project xwiki-commons
echo "*****************************"
echo -e "\033[1;32m Releasing xwiki-rendering\033[0m"
echo "*****************************"
# release_project xwiki-rendering
echo "*****************************"
echo -e "\033[1;32m Releasing xwiki-platform\033[0m"
echo "*****************************"
release_project xwiki-platform
... - Fix the build if need be (you won't need to fix anything if you just want to release again for example).
- Re-launch the maven-release.sh script from /releases/xwiki-trunks to resume from where we left off:hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks$ ~/maven-release.sh
If the release:prepare is good and you need to restart only the release:perform
Going even further with the resuming of a failed build, in the case of release:perform, we can resume from the actual failed module. To do so, we must make some additional temporary modifications to the release script, besides the one described above which isolates the script to release only the currently failed module.
Comment out the cleanup steps done in the release_project method:
cd $1
# pre_cleanup
# update_sources
# check_branch
# create_release_branch
# pre_update_parent_versions
release_maven
post_update_parent_versions
push_release
...
Comment out the release:prepare step in the release_maven method and add the -rf group:artefactId parameter inside the -Darguments="..." with the module you want to resume the release from. See an example below where we are resuming from org.xwiki.platform:xwiki-platform-distribution-war.
DB_PROFILE=hsqldb
echo -e "\033[0;32m* release:prepare\033[0m"
# mvn release:prepare -DpushChanges=false -DlocalCheckout=true -DreleaseVersion=${VERSION} -DdevelopmentVersion=${NEXT_SNAPSHOT_VERSION} -Dtag=${TAG_NAME} -DautoVersionSubmodules=true -Phsqldb,mysql,pgsql,derby,jetty,glassfish,legacy,integration-tests,office-tests,standalone -Darguments="-N ${TEST_SKIP}" ${TEST_SKIP} || exit -2
echo -e "\033[0;32m* release:perform\033[0m"
mvn release:perform -DpushChanges=false -DlocalCheckout=true -P${DB_PROFILE},jetty,legacy,integration-tests,office-tests,standalone ${TEST_SKIP} -Darguments="-P${DB_PROFILE},jetty,legacy,integration-tests,office-tests ${TEST_SKIP} -Dgpg.passphrase='${GPG_PASSPHRASE}' -Dxwiki.checkstyle.skip=true -rf org.xwiki.platform:xwiki-platform-distribution-war" -Dgpg.passphrase="${GPG_PASSPHRASE}" || exit -2
echo -e "\033[0;32m* Creating GPG-signed tag\033[0m"
...
Manually set the git tag name that will be used for GPG signing:
export TAG_NAME="xwiki-platform-8.3-milestone-2"
Run the release script to resume the release of the current project:
Generate Code Contributors list
- Automatically, on the agent:## list_contributors.sh <start_version> <end_version>
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks$ ~/list_contributors.sh 10.4 10.5-rc-1 - Manually (on the agent or locally, if the automatic option fails):
- Collect the list of code contributors for this release by running the following command on each branch on each repository that has been released (xwiki-commons, xwiki-rendering, xwiki-platform):> git fetch --tags
> git log --pretty=format:"%an" <previousTag>..<nextTag> | sort -u
## Example:
> git log --pretty=format:"%an" xwiki-commons-10.4..xwiki-commons-10.5-rc1 | sort -u
> git log --pretty=format:"%an" xwiki-rendering-10.4..xwiki-rendering-10.5-rc1 | sort -u
> git log --pretty=format:"%an" xwiki-platform-10.4..xwiki-platform-10.5-rc1 | sort -u
## For **final versions**, always compare with the previous final, not with the previous RC:
> git log --pretty=format:"%an" xwiki-platform-10.4..xwiki-platform-10.5 | sort -u
- Collect the list of code contributors for this release by running the following command on each branch on each repository that has been released (xwiki-commons, xwiki-rendering, xwiki-platform):
- Try to remove duplicates and infrastructure users (like xwikirogci or XWiki) and sort alphabetically.
- Update the Release Notes in the Credits section with the contributors list.
Handle Backward Compatibility Actions
Update Backward Compatibility in the release note
The Backward Compatibility report is automatically generated in the Release Notes page by the {{backwardCompatiblityReport version="<version>"/}} macro which does the following:
- Gets the revapi ignores from the pom.xml of xwiki-commons, xwiki-rendering and xwiki-platform GitHub repositories and store them in an xobject in the Release Note page. If you wish to regenerate the content from GitHub just delete this xobject.
- Displays the ignores
Update the backward compatibility setup in the build
If you're releasing a final version, perform the following operations on the agent or on your local machine:
- Automatically, on the agent:## backward_compatibility_cleanup.sh <new_version>
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks$ ~/backward_compatibility_cleanup.sh 10.5 - Manually (on the agent or locally, if the automatic option fails):
- Checkout master for xwiki-commons and edit the top level pom.xml to update the xwiki.compatibility.previous.version. Set the value to be your release you've just done.## Replace the previous version with the new version (e.g. 10.4 with 10.5).
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks/xwiki-commons$ git checkout master
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks/xwiki-commons$ sed -i 's/<xwiki.compatibility.previous.version>.*</<xwiki.compatibility.previous.version>10.5</' pom.xml
## Double check that it has been updated
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks/xwiki-commons$ git diff
## Commit and push
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks/xwiki-commons$ git commit -a -m "[release] Updated compatibility previous version to the one just released."
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks/xwiki-commons$ git push origin master - On master, you have to remove ignores from 3 locations and then commit & push the changes (git commit -a -m "[Misc] Removed revapi ignores from the previous version"):
- xwiki-commons/xwiki-commons-core/pom.xml
- xwiki-rendering/pom.xml
- xwiki-platform/xwiki-platform-core/pom.xml
- If you are doing this on the agent, you can use nano to open the files and use CTRL+W to search for revapi. Once you have found the JSON formatted ignores, select all the lines you want to delete by using CTRL+SHIFT+6 and press CTRL+K to delete the selected lines. Exit the exitor with CTRL+X, y, ENTER (when asked if the changes should be saved) and commit the changes for each repository.
- Checkout master for xwiki-commons and edit the top level pom.xml to update the xwiki.compatibility.previous.version. Set the value to be your release you've just done.
Make sure the xwiki.compatibility.previous.version value is changed also on the branch from where you've released, and that the ignores are cleaned too from the branch, since future releases (ex. 10.5.1) are final releases too.
Clean up identity
hudsonagent@vagent-1-1-dev:~/releases/xwiki-trunks$ cd
hudsonagent@vagent-1-1-dev:~$ cp .gitconfig.default .gitconfig
# GPG key
hudsonagent@vagent-1-1-dev:~$ gpg --delete-secret-and-public-keys `gpg --list-secret-keys | grep ^sec | cut -d/ -f2 | cut '-d ' -f1`
# Local changes to the release-scripts (maven-release.sh, user/password in release-translations.sh, etc.)
hudsonagent@vagent-1-1-dev:~$ cd xwiki-dev-tools
# Double check that you are not removing any improvement you have made to the release scripts that should be committed instead ;)
hudsonagent@vagent-1-1-dev:~/xwiki-dev-tools$ git status
hudsonagent@vagent-1-1-dev:~/xwiki-dev-tools$ git diff
hudsonagent@vagent-1-1-dev:~/xwiki-dev-tools$ git checkout .
Then don't forget to remove the Agent's SSH Key from your Github Account.
Blog post on xwiki.org
Create a new blog post on xwiki.org. You can have a look at the previous release blog posts for inspiration.
Update Wikimatrix
This step must be performed only for the latest final releases (not bugfixes or RCs).
If it's the first time please register on https://www.wikimatrix.org and ask someone (Thomas for example) to get admin access to XWiki project for you.
Note: The only exception when a bugfix would be added here is when it would be done for the latest released final version (e.g. latest final is 10.2 and the bugfix is 10.2.1).
Forum Announcement
- Add a new post to the users forum on https://forum.xwiki.org/c/News
- Example post:
- Subject: XWiki <version> released
- Content:The XWiki development team is proud to announce the availability of XWiki <version>.
<short summary>
You can download it here: https://www.xwiki.org/xwiki/bin/view/Main/Download
Make sure to review the release notes:
https://www.xwiki.org/xwiki/bin/view/ReleaseNotes/Data/XWiki/<short version>
Thanks for your support
-The XWiki dev team - Category: News & Events
- Tags: release
- Subject & Content: Same as for mail, but drop the [ANN] prefix
Announce on Twitter
To announce the new release on Twitter using XWiki.org`s account. Use the following type of text:
Rebuild Debian Distribution
The XWiki Debian distribution is automatically rebuilt every day using a crontab on maven.xwiki.org.
However, in order for the Debian distribution to be ready when we announce a new version, it's a good thing to force rebuild it.
This is achieved by logging on maven.xwiki.org with the maven user. Then run the script.
This start the index update in the background, you can leave.
Force Extensions Update
In order to have up to date versions on https://extensions.xwiki.org you should trigger the Failed to execute the [velocity] macro. Cause: [The execution of the [velocity] script macro is not allowed in [dev:ReleasePlans.ReleasePlanHelp]. Check the rights of its last author or the parameters if it's rendered from another script.]. Click on this message for details.
(from the first node of the cluster).
However before you do so, make sure to mute the IRC channel to not spam it:
And don't forget to unmute it once the release is over:
Update Docker
See documentation in the Docker GitHub project's README.