Changes for page Weblate Configuration
Last modified by Simon Urli on 2021/09/17 10:28
Change comment:
There is no comment for this version
Summary
-
Page properties (2 modified, 0 added, 0 removed)
Details
- Page properties
-
- Author
-
... ... @@ -1,1 +1,1 @@ 1 -xwiki:XWiki. ThomasMortagne1 +xwiki:XWiki.surli - Content
-
... ... @@ -1,223 +1,9 @@ 1 1 [[Weblate>>https://weblate.org/en/]] is an open source web-based translation tool written in Python. We use it as the implementation behind [[l10n.xwiki.org>>http://l10n.xwiki.org]]. 2 +This page documents the operations contributors are able to perform with Weblate UI. For more information about maintaining Weblate, [[check this page>>doc:xwiki:XWiki.Maintenance.Weblate.WebHome]]. 2 2 3 3 {{toc/}} 4 4 5 -= How to install = 6 6 7 -= =Prerequisite==7 += Add new component = 8 8 9 -* Python (2.7 in our setup) 10 -* Python virtualenv 11 -* Dependencies [[listed here>>https://docs.weblate.org/en/latest/admin/install.html#python-dependencies]] 12 -* Postgresql 13 -* Memcached 14 -* [[hub>>https://hub.github.com/]] 15 - 16 -__Quick setup:__ 17 - 18 -For debian users: {{box}}apt-get install memcached golang postgresql build-essential python virtualenv zlib1g-dev{{/box}} 19 -Python dependencies: 20 - 21 -{{code}} 22 -pip install --upgrade pip 23 -pip install -U setuptools 24 -pip install Babel certifi chardet defusedxml diff-match-patch Django django-appconf django-compressor django-crispy-forms djangorestframework filelock idna lxml oauthlib phply Pillow ply psycopg2 psycopg2-binary pydns PyJWT pyLibravatar python-bidi python-dateutil python-memcached python-openid pytz pyuca PyYAML rcssmin requests requests-oauthlib rjsmin setuptools siphashc six social-auth-app-django social-auth-core translate-toolkit ua-parser urllib3 user-agents Whoosh Weblate 25 -{{/code}} 26 - 27 - 28 -== Installation == 29 - 30 -Most of the installation process can be found on the [[official documentation>>https://docs.weblate.org/en/latest/admin/install.html]]. We chose the [[virtualenv method>>https://docs.weblate.org/en/latest/admin/install.html#installing-in-virtualenv]] to install all the python dependencies. 31 - 32 -Let's call (% class="mark" %)##$VIRTUALENV##(%%) the path of the virtual environment (it could be {{box}}~~/weblate{{/box}} for instance). 33 - 34 -After the installation, you can find weblate's files and other installed modules at {{box}}$VIRTUALENV/local/lib/python2.7/site-packages/{{/box}} (your python version may vary). 35 - 36 -Note that if you use virtualenv, you'll have to use the {{box}}weblate{{/box}} command instead of {{box}}./manage.py{{/box}} that you may found on weblate's documentation. 37 - 38 -== Configuration == 39 - 40 -Most of the configuration is made by editing the file {{box}}$VIRTUALENV/local/lib/python2.7/site-packages/weblate/settings.py{{/box}}. You'll find a sample configuration [[here>>https://docs.weblate.org/en/latest/admin/sample.html#sample-configurationhere]]. 41 - 42 -Here is what you can add to allow authentication using [[xwiki.org>>http://www.xwiki.org/]] credentials and what you should add to include the required scripts: 43 - 44 -{{code language="python"}} 45 -... 46 -from social_core.backends.oauth import BaseOAuth2 47 -... 48 -AUTHENTICATION_BACKENDS = ( 49 - ... 50 - 'weblate.settings.XWikiOAuth2', 51 -) 52 - 53 -class XWikiOAuth2(BaseOAuth2): 54 - name = 'XWiki' 55 - ID_KEY = 'sub' 56 - XWIKI_URL = 'https://www.xwiki.org/xwiki' 57 - AUTHORIZATION_URL = '{0}/oidc/authorization'.format(XWIKI_URL) 58 - ACCESS_TOKEN_URL = '{0}/oidc/token'.format(XWIKI_URL) 59 - REDIRECT_STATE = False 60 - ACCESS_TOKEN_METHOD = 'POST' 61 - EXTRA_DATA = [ 62 - ('refresh_token', 'refresh_token', True), 63 - ] 64 - 65 - def get_user_details(self, response): 66 - username = response.get('sub').split('.')[-1] 67 - fullname, first_name, last_name = self.get_user_names(response.get('name')) 68 - return { 69 - 'username': username, 70 - 'email': response.get('email'), 71 - 'fullname': fullname, 72 - 'first_name': first_name, 73 - 'last_name': last_name 74 - } 75 - 76 - def user_data(self, access_token, *args, **kwargs): 77 - return self.get_json( 78 - '{0}/oidc/userinfo'.format(self.XWIKI_URL), 79 - params={'access_token': access_token} 80 - ) 81 - 82 -SOCIAL_AUTH_XWIKI_KEY = '[SOCIAL_AUTH_XWIKI_KEY]' 83 -... 84 -POST_UPDATE_SCRIPTS = ( 85 - '[xwiki-dev-tools]/weblate-scripts/post_update.sh', 86 -) 87 -POST_UPDATE_SCRIPTS = ( 88 - '[xwiki-dev-tools]/xwiki-dev-tools/weblate-scripts/post_update.sh', 89 -) 90 -POST_COMMIT_SCRIPTS = ( 91 - '[xwiki-dev-tools]/xwiki-dev-tools/weblate-scripts/post_commit.sh', 92 -) 93 -PRE_COMMIT_SCRIPTS = ( 94 - '[xwiki-dev-tools]/xwiki-dev-tools/weblate-scripts/pre_commit.sh', 95 -) 96 -{{/code}} 97 - 98 -You should replace this parameters: 99 - 100 -* ##[SOCIAL_AUTH_XWIKI_KEY]##: choose a unique string 101 -* ##[xwiki-dev-tools]##: local path of [[https:~~/~~/github.com/xwiki/xwiki-dev-tools>>https://github.com/xwiki/xwiki-dev-tools]] 102 - 103 -Generate a secret key by running {{box}}$VIRTUALENV/share/weblate/examples/generate-secret-key{{/box}}. 104 - 105 -In order for the settings to be found, you'll have to set the ##DJANGO_SETTINGS_MODULE## environment variable. You can do it by adding {{box}}export DJANGO_SETTINGS_MODULE=weblate.settings{{/box}} at the end of {{box}}$VIRTUALENV/bin/activate{{/box}} file. 106 - 107 -Other useful configurations can be found [[here>>https://docs.weblate.org/en/latest/admin/install.html#production-setup]]. 108 - 109 -== Database configuration == 110 - 111 -Postgresql can be configured by following [[this steps>>https://docs.weblate.org/en/latest/admin/install.html#postgresql]]. 112 - 113 -== Github configuration == 114 - 115 -You can configure Weblate to use a github account to make pull requests as [[described here>>https://docs.weblate.org/en/latest/vcs.html#hub-setup]] (##xwikiorgci## can be used). 116 - 117 -= Running Weblate = 118 - 119 -Make sure you have sourced the virtualenv activate script ({{box}}source $VIRTUALENV/bin/activate{{/box}}) and run {{box}}weblate runserver{{/box}}. 120 - 121 -= How to import translation files manually = 122 - 123 -Login with an admin account to your weblate instance. You can generate one by running {{box}}weblate createadmin{{/box}}. 124 - 125 -Go to the administration panel by clicking the [[image:image.png||height="16" width="17"]] icon or directly by going to ##/admin##. 126 - 127 -You'll need to create a project frst. We usually create one project for one git repository, for instance we have an ##XWiki Platform## project. 128 - 129 -== Create the main component == 130 - 131 -You have to create the main component which will be the base of other components. A component is linked to a base translation file and all the related language files (e.g. ##ApplicationResources.properties## is the base translation file and ##ApplicationResources_fr.properties## is a related language file). This main component will be used to clone the git repository so that other components will use a link to this repository instead of making duplicates. 132 - 133 -Here is a configuration example of a main component: 134 - 135 -* Component name: **Help.SupportPanel.Content** 136 -* URL slug: **help-supportpanel-content** 137 -* Project: **XWiki Platform** 138 -* Version control system: **Github** 139 -* Source code repository: **https:~/~/github.com/xwiki/xwiki-platform** 140 -* Repository push URL: **https:~/~/github.com/xwiki/xwiki-platform** 141 -* File mask: **.translation/xwiki-platform-core/xwiki-platform-help/xwiki-platform-help-ui/src/main/resources/Help/SupportPanel/Content_*.properties** 142 -* Monolingual base language file: **.translation/xwiki-platform-core/xwiki-platform-help/xwiki-platform-help-ui/src/main/resources/Help/SupportPanel/Content_en.properties** 143 -* File format: **Java Properties (UTF-8)** 144 -* Additional commit files: 145 -** **xwiki-platform-core/xwiki-platform-help/xwiki-platform-help-ui/src/main/resources/Help/SupportPanel/Content.xml** 146 -** **xwiki-platform-core/xwiki-platform-help/xwiki-platform-help-ui/src/main/resources/Help/SupportPanel/Content.%(language)s.xml** 147 -* Post-update script: **{{scm path="xwiki-dev-tools/weblate-scripts/post_update.sh"}}post_update.sh{{/scm}}** 148 -* Pre-commit script: **{{scm path="xwiki-dev-tools/weblate-scripts/pre_commit.sh"}}pre_commit.sh{{/scm}}** 149 -* Post-commit script: **{{scm path="xwiki-dev-tools/weblate-scripts/post_commit.sh"}}post_commit.sh{{/scm}}** 150 - 151 -All files of this list as deployed at [[l10n.xwiki.org>>https://l10n.xwiki.org/]] are in the {{scm project="xwiki-dev-tools"}}xwiki-dev-tools repository{{/scm}}. 152 - 153 -After saving this component, Weblate will clone the git repository in {{box}}$VIRTUALENV/lib/python2.7/site-packages/data/vcs/xwiki-platform/help-supportpanel-content{{/box}}. An error should occur saying that the monolingual template file was not found. You need to run the (% class="mark" %)post_update.sh(%%) script a first time to fix this issue. For that, you can run {{box}}WL_PATH=$VIRTUALENV/local/lib/python2.7/site-packages/data/vcs/xwiki-platform/supportpanel xwiki-dev-tools/weblate-scripts/post_update.sh{{/box}}. See the [[readme file>>https://github.com/xwiki/xwiki-dev-tools/blob/master/weblate-scripts/README.md]] of the scripts repository to understand how it works. 154 - 155 -== Create other components == 156 - 157 -Here is an example of another component based on the previous one: 158 - 159 -* Component name: **AnnotationCode.Translations** 160 -* URL slug: **annotationcode-translations** 161 -* Project: **XWiki Platform** 162 -* Version control system: **Github** 163 -* Source code repository: **weblate:~/~/xwiki-platform/help-supportpanel-content** 164 -* Repository push URL: **none** 165 -* File mask: **.translation/xwiki-platform-core/xwiki-platform-annotations/xwiki-platform-annotation-ui/src/main/resources/AnnotationCode/Translations_*.properties** 166 -* Monolingual base language file: **.translation/xwiki-platform-core/xwiki-platform-annotations/xwiki-platform-annotation-ui/src/main/resources/AnnotationCode/Translations_en.properties** 167 -* File format: **Java Properties (UTF-8)** 168 -* Additional commit files: 169 -** **xwiki-platform-core/xwiki-platform-annotations/xwiki-platform-annotation-ui/src/main/resources/AnnotationCode/Translations.xml** 170 -** **xwiki-platform-core/xwiki-platform-annotations/xwiki-platform-annotation-ui/src/main/resources/AnnotationCode/Translations.%(language)s.xml** 171 -* Post-update script: **none** 172 -* Pre-commit script: **pre_commit.sh** 173 -* Post-commit script: **post_commit.sh** 174 - 175 -= How to import translation files automatically = 176 - 177 -There are many components to be created, that's why the (% class="mark" %)generate_components.py(%%) script has been written to generate a json file with all the components to be created, based on some lists of translation files. 178 -The script can be found in the [[xwiki-dev-tools project>>https://github.com/xwiki/xwiki-dev-tools]]: {{box}}xwiki-dev-tools/weblate-scripts/generate_components.py{{/box}} 179 -A list of translation file has a name like this one `translation_list_xwiki-commons.txt`, where `xwiki-commons` is the name of the project, and looks like this: 180 - 181 -(% class="box" %) 182 -((( 183 -#Format: Name of the component; Relative path to the translation file; Repository url 184 - xwiki-commons-extension-api; xwiki-commons-core/xwiki-commons-extension/xwiki-commons-extension-api/src/main/resources/ApplicationResources.properties; https:~/~/github.com/xwiki/xwiki-commons 185 - xwiki-commons-job; xwiki-commons-core/xwiki-commons-job/src/main/resources/ApplicationResources.properties; https:~/~/github.com/xwiki/xwiki-commons 186 -))) 187 - 188 -Running the script will generate the ##components_xwiki-commons.json## file. It can be imported by running {{box}}weblate import_json components_xwiki-commons.json ~-~-project xwiki-commons{{/box}}. 189 - 190 -== Importing new translation files (full procedure) == 191 - 192 -If you just want to add new translation files to an existing project, you'll have to change two files and run two commands. 193 -Let's say you want to add a ##.properties## translation file from the ##xwiki-platform## project. In the [[xwiki-dev-tools project>>https://github.com/xwiki/xwiki-dev-tools]], you'll have to change the files ##xwiki-dev-tools/weblate-scripts/translation_list_xwiki-platform.txt## and ##xwiki-dev-tools/weblate-scripts/translation_properties.txt##. 194 - 195 -1. Add at the end of ##translation_list_xwiki-platform.txt## a line with the following format: {{box}}Name of the component; Relative path to the translation file; Repository url{{/box}} 196 -For instance: {{box}}Help.SupportPanel.Content; xwiki-platform-core/xwiki-platform-help/xwiki-platform-help-ui/src/main/resources/Help/SupportPanel/Content.xml; https://github.com/xwiki/xwiki-platform{{/box}} 197 -1. Add at the end of ##translation_properties.txt## a line with the relative path to the translation files. 198 -For instance: {{box}}xwiki-platform-core/xwiki-platform-template/xwiki-platform-template-api/src/main/resources/ApplicationResources.properties{{/box}} 199 -1. Run the {{box}}generate_components.py{{/box}} script to generate ##components_xwiki-platform.json## 200 -1. Run the {{box}}call_update.py{{/box}} script with the full path of the weblate VCS, the name of the project, and the Git link of the project. 201 -For instance: {{code}}./call_updates.py /home/weblate/weblate/lib/python2.7/site-packages/data/vcs --project xwiki-platform --component https://github.com/xwiki/xwiki-platform{{/code}} 202 -1. Run the command {{code}}weblate import_json components_xwiki-platform.json --project xwiki-platform --ignore{{/code}} to finally import the new components (translation files). 203 -1. Run the commands to install the 3 different addons (pre-commit, post-commit and post-update) to the new component: 204 - 205 -* {{code}}weblate install_addon --addon xwiki.pre_commit xwiki-platform/jcaptcha-ui{{/code}} 206 -* {{code}}weblate install_addon --addon xwiki.post_commit xwiki-platform/jcaptcha-ui{{/code}} 207 -* {{code}}weblate install_addon --addon xwiki.post_update xwiki-platform/jcaptcha-ui{{/code}} 208 - 209 -{{warning}} 210 -Note that all commands starting with {{code}}weblate{{/code}} are done inside the weblate virtualenv. To enter in this virtualenv just run {{box}}source ~/weblate/bin/activate{{/box}}. 211 -{{/warning}} 212 - 213 -== Importing new translation files (simplified procedure) == 214 - 215 -Instead of relying on all steps described above, it's heavily recommended to directly used the ##[[add_new_component.py>>https://github.com/xwiki/xwiki-dev-tools/blob/master/weblate-scripts/add_new_component.py]] script directly on Weblate machine which will automate most of the work.## 216 - 217 -= How to upgrade Weblate = 218 - 219 -The whole process can be found [[here>>https://docs.weblate.org/en/latest/admin/upgrade.html]]. 220 - 221 -= Miscellaneous = 222 - 223 -If the offloading indexing (OFFLOAD_INDEXING parameter) is activated (which should be), you'll have to run this command frequently (it can be a cron job) to update the index: {{box}}weblate update_index{{/box}}. 9 +Each source translation file on a git repository is handled by a Weblate component. So whenever a committer adds a new source translation file, she should also creates the corresponding Weblate component.