Translate devguide

[rffontenelle]

I would like to suggest enabling translation of the Python Developer's Guide (devguide), and setting it up in a translation platform.

Below I explain in more detail why this brings several benefits for both the devguide itself and the community. I created the pull request #1011 as an initial suggestion, but which surely can be adjusted afterwards.

Why?

Translating a documentation brings several benefits:

• Improved accessibility: Translating the devguide into different languages would make it accessible to a wider audience, including those who do not speak English natively and are improving it.

• Increased engagement: Localizing the devguide can help to increase engagement with the community of voluntary contributors, as it provides an opportunity for them to contribute to the project in a meaningful way. This can help to foster a sense of ownership and belonging among contributors, which can lead to increased participation and collaboration. For instance, Weblate made their documentation available for translation and several typos were found/reported along with other improvements from contributors initially engaged in translations.

This will not result in too much extra work if combined with a solution that automatizes translation updates and a translation platform.

How?

sphinx-build has a builder named gettext which extracts source strings from reStructuredText-formatted files and store then in Gettext POT files, and sphinx-intl's update command updates PO files with newer POT files. If the language does not exist, sphinx-intl's build command together with the target language must be passed. See Internationalization at Sphinx docs for more information.

Solution: a target in the Makefile that calls sphinx-build gettext and then sphinx-intl update (with respective flags)

A GitHub Actions workflow can be set to run this Makefile target whenever there is a push event in devguide repository, and commit the updated translation files.

Workflow?

Whenever a push to branch main is made, the translation files are updated. No manual action required here.

There are two methods for the translators to contribute with translations:

1. Filing pull requests with the translation edited
2. Contributing to a translation platform (Weblate, Transifex, etc.)

The first method requires knowledge of what local translation tool to use (emacs, gtranslator, poedit, etc.), how to work with Git (translate, commit, push) and how to open pull requests. Not the end of the world, but includes a technical requirement that slowdown contributions. Also, translators are not notified whenever the PO files are updated with latest changes in the strings extracted from the docs.

The second method requires setting up a project in the translation platform, but is much more a translator-friendly solution and can notify the translation whenever the translation file is updated.

The method for pushing contributions from the translation platform to devguide repository depends one the choice: automatically create/update a pull request (requires maintainer merge acceptance), automatically push to the repository (zero intervention of a maintainer), only manually pushes (maintainer has to remember to fetch translations from the translation platform).

Decision: Single translation file vs. per-rst files

sphinx-build gettext has this gettext_compact option that once set with 0 or False will generate a PO file for each reST file. This is great for smaller PO files, although there will be a great number of them.

Setting gettext_compact with True or a string (e.g. 'devguide'), it will concatenate all strings into a single file per-language. This is great for using in a translation platform as a single translation component (easier flow), although it will have a great total amount of strings.

Personally, I don't see a problem in using a single translation file, considering that this documentation is not too big. It is a different scenario than huge docs like Django's or Python's. But I'm open to debate.

Translation platform

Weblate is a free and open-source translation platform that eases software and docs translation, providing a web-based interface for translation to collaborate.

It features syntax-highlight, translation memory, machine translation integration and quality checks. Its main developer is very committed with the open source ecosystem, also maintaining translation-toolkit just for an example.

Weblate has a hosted instance with paid plans, but there is a "libre hosting" plan for software that meets the free and open source licensing criteria -- devguide meets them.

P.s. Python Docs is currently translated by some language teams in Transifex (others use Git workflow). Python Docs project has almost 54k source strings, so it doesn't meet the criteria of Weblate's maximum number of 10K source strings in the libre hosting plan. However this does not apply to devguide, which has less than 4K strings!

My suggestion

As I said before, I created the pull request #1011 as an initial suggestion. I did not consider the use of Weblate in it. It is ready for users to file pull requests for adding and updating translations.

If we're going to use Weblate as a translation platform, I can adjust the pull request. I'd also be more than happy to help on this transition.

[CAM-Gerlach]

Hey @rffontenelle , thanks for the very detailed and thorough proposal as well as the PR implementing it!

I do have some questions about this:

• Unlike the Python Documentation or other user-facing docs, unfortunately I'm not sure what types of contributions discussed in the devguide wouldn't require at least satisfactory English proficiency, and thus would benefit from a translated version:
  • Reporting, triaging and otherwise helping with issues generally necessitates being able to read and write English, as the common language of communication on Python's issue tracker
  • Contributing and reviewing code requires the ability to communicate in English in issues, pull requests, reviews, comments, commit messages, etc
  • The upstream Python documentation is written and edited in English, so contributors (and particularly reviewers) still need a command of the language
  • Translators need to be able to read English as well as the language(s) they are translating to in order to be able to accurately translate the source material
  • PEPs are written and discussed in English as well
• Who would actually perform the translations, check them for accuracy and keep them up to date over time? It seems to me that limited translator bandwidth might be better spent on the Python documentation, website and other user-facing resources, particularly given the above, and I'd worry these might get out of date and give users inaccurate guidance. It also imposes a much greater downstream cost to updating, improving and expanding the devguide, due to the need to expand/update all of the translations as well.
• Assuming we'd call upon our existing docs translators, we'd presumably want to use a workflow/platform that they are already using, as opposed to adopting a brand new service. Therefore, while Weblate sounds like a nice platform, I'm not sure it would be ideal from that perspective, since I'm not aware of any core Python project actively using it at present (though one might be).
• In addition to the backend changes in Add translation (l10n) infrastructure to devguide #1011 , we would also need to add support for a translation menu in the theme, and add support on our hosting serve (RTD?)—as of now, on Add translation (l10n) infrastructure to devguide #1011 , the translated versions don't appear to be publicly exposed anywhere I could find. However, I assume that's not too hard to fix.
• The theme text is not itself translated, see Translations for the theme pradyunsg/furo#434 for more.

@JulienPalard as the leader of the docs translations effort, what do you think about this?

[ezio-melotti]

I agree with most of what @CAM-Gerlach said. There are some pages that might benefit from a translation (e.g. the git bootcamp page), but otherwise I'm not sure it's worth the effort and the maintenance cost.

Assuming we'd call upon our existing docs translators, we'd presumably want to use a workflow/platform that they are already using, as opposed to adopting a brand new service.

In addition, I'd like to see some effort in consolidating the tools used by the current translation teams, since I saw them not only using different tools, but also re-inventing slightly different variations of the same wheel for things like CI. This should be discussed in a separate issue, but if we do end up deciding to translate the devguide, I would like to start from there rather than inventing yet another wheel.

[rffontenelle]

@CAM-Gerlach: I'd love to see this translated, but I understand your point of view.

we would also need to add support for a translation menu in the theme

Indeed, this is an issue. RTD's default theme has a menu that shows the languages available, but that does not seem to be furo's case.

add support on our hosting serve (RTD?)—as of now, on Add translation (l10n) infrastructure to devguide #1011 , the translated versions don't appear to be publicly exposed anywhere I could find. However, I assume that's not too hard to fix.

I'm sorry. Actually, readthedocs (RTD)'s building pull requests feature applies to the language specified in the configuration. Since cpython-devguide is set with English, that's the language built in that pull request. That was useless for pull request, as you can see.

However, before setting up this pull requests, I set a playground in https://github.com/rffontenelle/devguide-pt-br with a totally different structure think on a decentralized translation workflow (like Python Docs workflow, where each team has its own repository etc.) and also set a RTD project in https://devguide-pt-br.readthedocs.io/pt_BR/latest/. That one is in Portuguese (not all, since I haven't finished the translation).

If the translation is deployed, one could set up a cpython-devguide-pt-br in RTD, and then configure it to be a translation of the cpython-devguide RTD project. Once this done, a menu (which furo does not have) would link to the translations available.

The theme text is not itself translated, see Translations for the theme pradyunsg/furo#434 for more.

Yeah, I'm planning to take a look on how I can help on it. Make some learning from sphinx_rtd_theme can help.

@ezio-melotti I agree with you regarding consolidating tools. Having one solution for all teams would reduce maintenance tasks and would help new translation teams to start contributing. Weblate would be great 😝. But yeah, that's not the place for this discussion.

[rffontenelle]

I believe the discussion here is done and a consensus has been reached that this is not something to be implemented, so I'll be closing this. Nonetheless, thanks for your the valuable feedbacks.

[CAM-Gerlach]

(Changing the close reason to the appropriate one matching the above)