Release Plan Help

Last modified by Thomas Mortagne on 2014/10/17 13:30

Contains instructions to perform various release steps in the Release Plans.

Initial CLIRR check

This initial step checks if the previous release cycle, or more recent bugfix releases done since, (1) have properly set the CLIRR xwiki.clirr.previous.version in the top level POM of xwiki-commons AND (2) that there are no leftover CLIRR excludes from the previous releases.

(1) For example, if the current release to be made is for 4.2-milestone-3 and, in the meanwhile, there have been released bugfix versions for 4.1 (4.1.4 being the latest released), you need to ensure that the value for xwiki.clirr.previous.version is 4.1.4.

(2) When looking at the current CLIRR excludes in each top level project (xwiki-platform, xwiki-commons, etc.), you will generally see comments like "<!-- Remove the following excludes after we release the current version as final -->" at the beginning of a block/group of excludes. However, there might also be some leftover excludes, from previous releases that were unintentionally preserved and must be removed. These cases should be identified based on the comment or by using GitHub's blame view and looking at the time they were introduced, compared to the current version.

Note: to help you find pom.xml files with CLIRR excludes you can use:

xwiki-trunksgrep -H "clirr-maven-plugin" . -R --include=pom.xml --exclude-dir=*target*

Log on Release machine

# Get on the maven CI master machine
me@home:~ssh maven@maven.xwiki.org

# Get on the agent machine
maven@maven:~a1-1

Set up your identity

  • Configure your Github profile to add the SSH key of the agent so that the commits can be pushed to Github with your user (To get the agent SSH key, do cat ~/.ssh/id_dsa.pub). To verify it works, do the following on the agent machine: ssh -T git@github.com. It should reply with Hi <your user id>! You've successfully authenticated, but GitHub does not provide shell access..
  • Create your own GPG key if you don't already have one: gpg --gen-key
  • Import your GPG key:
    • On your local computer run: gpg --list-secret-keys
    • Export your key in a secret.key file: gpg --export-secret-keys «keyID» > secret.key where keyID is the 8 characters long hexadecimal string on the sec line
    • Copy the secret.key file to the agent machine (you may need to copy to maven.xwiki.org first and then to agent-1-1)
      • From your local computer, upload the key to maven.xwiki.org: scp secret.key maven@maven.xwiki.org:/home/maven
      • From maven.xwiki.org (/home/maven): scp secret.key hudsonagent@192.168.1.19:
        • You can remove the key from maven.xwiki.org after you have uploaded it to agent-1-1.
        • In case the IP address of the agent written above is no longer correct, just connect to the maven CI master machine (as shown in the Log on Release machine section above), run the command alias and get the IP address from the line containing the "a1-1" alias (the one used to connect to the agent). It looks like alias a1-1='ssh hudsonagent@192.168.1.19'. After that, please update this page emoticon_smile
    • On the agent machine, run gpg --import secret.key
      • Don't leave the key file on the agent machine, run rm secret.key
  • You need to have a .gitconfig.<yourusername> file. You can do that by copying one of the existing gitconfig file
  • Make sure your gitconfig file has the key signingkey in the [user] section (the value is your key id which you can get with gpg --list-secret-keys).
  • Copy your .gitconfig.<yourusername> file as the main gitconfig file: cp .gitconfig.<yourusername> .gitconfig

Update Release scripts

# Update the release script
hudsonagent@ks365275:~cd xwiki-dev-tools ; git pull --rebase ; cd ..
Do a git status to verify that there are not any committed but not pushed stuff lying around...

Update Translations

When you release a .x version (bug fix release), be very careful when you update translations because the new ones might concern new stuff introduced in the master branch. The issue is that the translations wiki does not support branches.

Perform the following steps:

  • Edit the release-translations.sh file to set branch, if needed (e.g. do this only if you are performing a bugfix release like 6.1.1 on the stable-6.1.x branch and you are not working on master, which is 6.2-SNAPSHOT).:
    hudsonagent@ks365275:~vi release-translations.sh
  • Move to xwiki-trunks:
    hudsonagent@ks365275:~cd releases/xwiki-trunks/
  • Download the translations:
    hudsonagent@ks365275:~/releases/xwiki-trunks~/release-translations.sh
  • Review the changes for anything amiss, like spam in translations
  • Collect the list of updated languages and update the Release Notes

    In case there are new translation files for GWT modules (xwiki-platform-gwt-user or xwiki-platform-wysiwyg-client), those languages need to be added to the corresponding GWT module descriptors:

    ## For xwiki-platform-gwt-user
    vi xwiki-platform-core/xwiki-platform-gwt/xwiki-platform-gwt-user/src/main/resources/org/xwiki/gwt/user/User.gwt.xml
    ## For xwiki-platform-wysiwyg-client
    vi xwiki-platform-core/xwiki-platform-wysiwyg/xwiki-platform-wysiwyg-client/src/main/resources/org/xwiki/gwt/wysiwyg/Wysiwyg.gwt.xml

    ...and add the new language code to the existing list defined in the locale extended-property that looks similar to this:

    <extend-property name="locale" values="ca,cs,da,de,es,fr,it,lv,nl,pt_BR,ru,sk,sv,tr,zh,zh_TW" />
  • Commit the translation changes:
    hudsonagent@ks365275:~/releases/xwiki-trunks~/release-translations.sh commit
Be patient, this process takes some time

Build the release

Make sure you use the right version of Java for the branch you are releasing: ~/java symbolic link should target java7 for version >= XWiki 6.x and java6 for version < XWiki 6.x (for example version 5.4.3 is to be released with java6).
# Perform the release from the release sources
hudsonagent@ks365275:~/releases/xwiki-trunks~/maven-release.sh

When releasing a Release Candidate (e.g. 6.2-rc-1), make sure that, when asked What is the next SNAPSHOT version?, you do not increment the snapshot version (i.e. you answer 6.2-SNAPSHOT instead of 6.3-SNAPSHOT or whatever suggestion you have). This is because the next step will ask you to create a stable branch where the final version (if this is the last RC we`re doing) will be released and there we need to keep the SNAPSHOT version to the current one.

If you have created the stable branch and are asked What is the next master version?, you will now increment the snapshot version (i.e. answer 6.3-SNAPSHOT) so that the master branch can move to the next development version.

Tip: Because the build process is a lenghty one (currently around 2 hours), it is a good idea to use the screen linux command. This is most helpful in case your Internet connection or electrical power drops. However, it is not in any way mandatory to use screen during the release.

Tip: running the release script will first ask you for your GPG key passphrase. If you mistype or enter a wrong passphrase, the release will fail to perform. You can test your passphrase locally running the following command : 

echo "test" | gpg --no-use-agent -o /dev/null --local-user <KEYID> -as - && echo "Passphrase is correct"

If for a reason or another you've made a mistake typing your passphrase in, you will need to delete (locally on the agent machine) the release branch and created tag before you can start over again.

If you get the following error then type export GPG_TTY=`tty` and then retry:

Connection refused
pinentry-curses: no LC_CTYPE known - assuming UTF-8

Release Candidate

In case the version you've just build was a Release Candidate, there are 2 extra steps you need to perform, which are to update the parent version and version properties on the master branch and to update the continuous integration jobs to build the new stable version that you have just created trough the release of a RC version.

Update the parent version and version properties

This step is needed because master has now a new/incremented version but the parent (of top projects) and properties like commons.version are still using the old/previous version. To fix this, you need to run the following command:

# Replace 4.2-SNAPSHOT with the old snapshot version that you have just released as a RC and '4.3-SNAPSHOT' with the new snapshot version to be used for development.
hudsonagent@ks365275:~/releases/xwiki-trunksgrep "4.2-SNAPSHOT" . -R --exclude-dir=target --include=pom.xml --exclude-dir=target-eclipse -l | xargs -i@ sed -i 's/4.2-SNAPSHOT/4.3-SNAPSHOT/g' @

Now go trough each top level project(xwiki-commons, xwiki-platform, etc...) and commit and push the local changes, using the message [release] Incremented parent versions and/or version properties for the next development iteration.

hudsonagent@ks365275:~/releases/xwiki-trunkscd xwiki-commons
hudsonagent@ks365275:~/releases/xwiki-trunks/xwiki-commonsgit diff
hudsonagent@ks365275:~/releases/xwiki-trunks/xwiki-commonsgit commit -a -m "[release] Incremented parent versions and/or version properties for the next development iteration"
hudsonagent@ks365275:~/releases/xwiki-trunks/xwiki-commonsgit pull --rebase
hudsonagent@ks365275:~/releases/xwiki-trunks/xwiki-commonsgit push origin master
TODO
Try to automate this using the versions-maven-plugin

Update the continuous integration jobs

At this point, you have just created a new stable-x.y.z branch in the source repository where your version will achieve final version and will continue stabilizing trough bugfix versions, while the main development branch is done on the master branch.

However, the continuous integration jobs that are supposed to build the stable-x.y.z branch are not yet set up. They are still configured to build the previous stable branch (stable-x.y-1.z or stable-x-1.y.z). This means that you need to add the new jobs. Fortunately we have an automated script for this emoticon_wink

Go to http://ci.xwiki.org/scriptler/runScript?id=copy-jobs.groovy , fill the parameter values and click on Run at the bottom right. 

Make sure that the script properly copies build triggers. Inspect the execution logs of the script and fix any missing build triggers in the copied jobs.

In case of failure while building

In our examples below, we are assuming that the xwiki-platform project just failed during the release, while the xwiki-commons and xwiki-rendering were successfullt released.

Go to the corresponding top level module, clean up and fix the build problem:

hudsonagent@ks365275:~/releases/xwiki-trunkscd xwiki-platform
# Note: Make sure to set the VERSION environment variable to the version you are working to release, otherwise the restart-release script will fail.
hudsonagent@ks365275:~/releases/xwiki-trunksexport VERSION='4.2-milestone-3'
hudsonagent@ks365275:~/releases/xwiki-trunks/xwiki-platform~/restart-release.sh

To resume the build from the top level module that previously failed, you need to edit the maven-release.sh script and comment out the previously successfully released top level modules.

hudsonagent@ks365275:~/releases/xwiki-trunks/xwiki-platformcd ..
hudsonagent@ks365275:~/releases/xwiki-trunksvi ~/maven-release.sh

Go to the end of the maven-release.sh file and scroll up, until you reach this section:

# Wrapper function that calls release_project for each XWiki project in order: commons, rendering, platform, enterprise, manager.
# This is the main function that is called when running the script.
function release_all() {
...

Inside this method, comment out the previously successful top level projects and leave uncommented the top level projects that still need to be released, including the one we are resuming from. Remember that in our example, we are resuming from xwiki-platform but xwiki-commons and xwiki-rendering were already successfully released, so we are skipping them:

...
 echo              "*****************************"
 echo -e "\033[1;32m    Releasing xwiki-commons\033[0m"
 echo              "*****************************"
#  release_project xwiki-commons
 echo              "*****************************"
 echo -e "\033[1;32m    Releasing xwiki-rendering\033[0m"
 echo              "*****************************"
#  release_project xwiki-rendering
 echo              "*****************************"
 echo -e "\033[1;32m    Releasing xwiki-platform\033[0m"
 echo              "*****************************"
  release_project xwiki-platform
 echo              "*****************************"
 echo -e "\033[1;32m    Releasing xwiki-enterprise\033[0m"
 echo              "*****************************"
  release_project xwiki-enterprise
...

Go back to the build step and re-launch the maven-release.sh script to resume from where we left off.

Clean up identity

# Git identity
hudsonagent@ks365275:~/releases/xwiki-trunkscd
hudsonagent@ks365275:~cp .gitconfig.default .gitconfig

# GPG key
hudsonagent@ks365275:~gpg --delete-secret-and-public-keys `gpg --list-secret-keys | grep ^sec | cut -d/ -f2 | cut '-d ' -f1`

# Local changes to the release-scripts (maven-release.sh, user/password in release-translations.sh, etc.)
hudsonagent@ks365275:~cd xwiki-dev-tools
# Double check that you are not removing any improvement you have made to the release scripts that should be committed instead ;)
hudsonagent@ks365275:~/xwiki-dev-toolsgit status
hudsonagent@ks365275:~/xwiki-dev-toolsgit diff
hudsonagent@ks365275:~/xwiki-dev-toolsgit checkout .

Then don't forget to remove the Agent SSH from your Github Account.

Generate CLIRR Report

The CLIRR report is always recorded in the release notes compared to the most recent released final version. This means that in all the non-final versions that we release, we will not report the differences with the previously released milestone version (ex: diff between 4.2M2 and 4.2M3), but we will always compare with the latest final release (ex: 4.1.4) and we will make sure to explicitly mention it in the release notes.

This is different from what we do in the release notes for features, where we list differences even between non-final versions, but we believe that it is simpler for both the users and the developers to do it this way.

Automatically

2012-07-12: This is not working for the following reasons:

  • maven-release.sh sometimes fail to generate correct CLIRR reports
  • The comments in the POM files above the CLIRR excludes are lost BUT we need to copy them in the CLIRR report in the Release notes
  • The script doesn't update the previous CLIRR version to compare too
  • The CLIRR excludes are not removed
  • New format for CLIRR ignores is not yet supported by the automated script

You must use the Manual strategy for now (see below).

The maven-release.sh script run previously is supposed to have generated clirr.txt files at the root of each repository. Thus to get the full CLIRR report you would run:

# Collect API breakages (clirr reports)
hudsonagent@ks365275:~/releases/xwiki-trunkscat xwiki-commons/clirr.txt xwiki-rendering/clirr.txt xwiki-platform/clirr.txt xwiki-enterprise/clirr.txt

Manually

The following operations need to be performed on your local machine.
  • Checkout the tag you've just released for all Git repositories having java code (xwiki-commons, xwiki-rendering, xwiki-platform)
    cd xwiki-commons
    git fetch --tags
    # If you`re releasing 4.2-rc-1
    git checkout xwiki-commons-4.2-rc-1
  • For each repository root do the following on your local machine:
    • Find all CLIRR excludes:
      grep -l "<artifactId>clirr-maven-plugin</artifactId>" --include=pom.xml . -R --exclude-dir=*target*
    • For each POM file found, copy/paste the exclude and ignored lists somewhere for later and remove them, but keep the following 2 excludes:
       <exclude>**/internal/**</exclude>
       <exclude>**/test/**</exclude>
    • Get the CLIRR violations:
      mvn clirr:check -DfailOnError=false -DtextOutputFile=clirr.txt -Plegacy,integration-tests
      find . -name clirr.txt | xargs cat | grep ERROR | sed -r -e 's/(ERROR: [0-9]+: )|(\s+$)//g'
      find . -name clirr.txt -delete
      On some OS (like Mac OSX) use instead: find . -name clirr.txt | xargs cat | grep ERROR | sed  -E -e 's/(ERROR: [0-9]+: )|(\s+$)//g'
    • Copy the content of clirr.txt on the Release notes page and for each violation copy the explanation from the pom.xml's CLIRR section (where the excludes are located)
    • If this is not a final version (you`re releasing a milestone), discard your changes to the pom.xml files. Note: When releasing final versions, you don`t have to do this (you actually have to commit removals of exclusions you`ve just done in order to start fresh the new version)
      git reset --hard HEAD
  • If you are releasing a final version:
    • Checkout the xwiki-commons branch corresponding to your release (for example if you're releasing 4.1.3, checkout stable-4.1.x) and edit the top level pom.xml to update the xwiki.clirr.previous.version. Set the value to be your release you've just done.
    • Checkout master for xwiki-commons and edit the top level pom.xml to update the xwiki.clirr.previous.version. Set the value to be your release you've just done.

Push to OW2

  • Log on the Jenkins master machine: ssh maven@maven.xwiki.org
  • Go to the release script directory: cd xwiki-release-scripts
  • Create a Release notes file: vi releasenotes-<version>.txt (e.g. releasenotes-6.1-milestone-2.txt)
    This milestone contains a new mail API and module to replace the old mailsender plugin, Flamingo skin and Extension Manager improvements as well as various performances improvements and bug fixes.

    Full release notes available at http://www.xwiki.org/xwiki/bin/view/ReleaseNotes/ReleaseNotesXWiki61M2
  • Upload to OW2 with ./push-release.sh push_ow2 (Follow instructions)
    • In case there is a problem during the upload and some of the files are not uploaded correctly, you can repeat this step to properly upload all the files.
    • You will need an OW2 account with admin rights on the XWiki project for this step
    • You may experience an "Could not chdir to home directory ... No such file or directory" error message if your account is just minutes old. If so, give it about 10-20 minutes for the new account to properly be created. It may also help (though this is just guesswork based on personal experience) if you register a public SSH key in the account management. Repeat the upload step until the files are all uploaded.
  • Create the release on OW2 with: ./push-release.sh update_ow2 (Follow instructions) Since OW2 forge is a unusable crap there is a good chance it does not fully work so you will have to do it by hand on http://forge.ow2.org/project/admin/editpackages.php?group_id=170
  • Verify all is ok by going to the XWiki project page on OW2. Specifically, verify that the pushed files can be downloaded before continuing with the release process as otherwise end users will get an error when trying to download XWiki.
After update_ow2 is executed it takes several minutes before the files are available from the http://download.forge.objectweb.org/xwiki/.

News on OW2

  • Publish the news on OW2 with: ./push-release.sh announce_ow2 (Follow instructions)
  • Verify all is ok by going to the XWiki project page on OW2

Blog post on xwiki.org

Create a new blog post on xwiki.org. You can use the same text that you`ve used in the OW2 release notes or otherwise have a look at the previous release blog posts for inspiration.

In order to display the new blog post in the "Latest Blog Posts" sections of the xwiki.org homepage, you need to refresh the document cache that is currently used. To do that, edit the macro object's code, look for the "cache" macro call and modify the "id" parameter's value (for instance from "latestBlogPostsCache3" to "latestBlogPostsCache4"). This will trigger a refresh on the homepage and the new blog post will be listed.

Announcement Mail

  • Send the announcement email to the users and devs list
  • Example email:
    • Subject: [ANN] XWiki <version> released
    • Content:
      The XWiki development team is proud to announce the availability of XWiki <version>.
      <short summary>

      You can download it here: http://www.xwiki.org/xwiki/bin/view/Main/Download

      Make sure to review the release notes:
      http://www.xwiki.org/xwiki/bin/view/ReleaseNotes/ReleaseNotesXWiki<short version>

      Thanks
      -The XWiki dev team

Announce on Twitter

To announce the new release on Twitter using XWiki.org`s account, you need first to create the PURL.org link that will be published in the standard tweet. An example PURL looks like this: http://purl.org/xwiki/rn/51M2 

PURL

  • [FIRST TIME] Create an account on purl.org
  • [FIRST TIME] Ask for access from another committer to be able to create links under the /xwiki/rn/ path
  • Create the new PURL
    • Path: /xwiki/rn/<shortVersion> (ex: 42, 503, 51M2)
    • Target URL: <blogPostLink> (ex: http://www.xwiki.org/xwiki/bin/view/Blog/XWiki+51+Milestone+2+Released)
    • Maintainers IDs (one per line):
      <yourUserName>
      mflorea
      tmortagne
      SDUMITRIU

Twitter

Connect to maven.xwiki.org and run:

maven@maven:~cd ~/xwiki-release-scripts
maven@maven:~/xwiki-release-scripts./push-release.sh announce_twitter

The announce_twitter command will extract the proper PURL to use in the twitter message, based on the VERSION that you are releasing.

Publish to Maven Central

XWiki Commons and XWiki Rendering need to be published to Maven Central. To do so we use Sonatype's sync. The strategy is to upload to the XWiki Staging repo on Sonatype's Nexus, to check the staged repo and then to promote it.

In order to be able to upload to Sonatype's Nexus you'll need to create an account on http://issues.sonatype.org and then ask for publish rights (if you don't have them already) by posting a comment to https://issues.sonatype.org/browse/OSSRH-2099 .

  • Add the following in your settings.xml:
      <server>
        <id>sonatype-nexus-staging</id>
        <username>userid</username>
        <password>password</password>
      </server>
  • Add passphrase in settings.xml or pass it on command line using -Dgpg.passphrase=...:
    ...
       <profile>
         <id>xwiki</id>
         <properties>
           <gpg.passphrase>password</gpg.passphrase>
         </properties>
    ...
     
  • in xwiki-commons and in xwiki-rendering, after having checked out the correct tag, run:
     mvn clean deploy -Prelease,legacy,standalone,integration-tests -DaltDeploymentRepository=sonatype-nexus-staging::default::https://oss.sonatype.org/service/local/staging/deploy/maven2 -DskipTests=true
  • Publish your public GPG key, if you have not already done so: gpg --keyserver hkp://pool.sks-keyservers.net --send-key <keyID>
  • Then log in https://oss.sonatype.org/ and perform the release steps as indicated on Sonatype's synchro doc.
Tags:
Created by Vincent Massol on 2012/07/13 15:39
    

Get Connected