Merge pull request #553 from carpentries/update-vignettes

Update translation and data flow vignettes

NEWS.md

@@ -12,7 +12,10 @@
 
 * A new vignette `vignette("translation", package = "sandpaper")` describes how
   translation of template components works and how to submit new/update
-  translations.
+  translations (added: @zkamvar, #546).
+- A new vignette about data flow `vignette("data-flow", package = "sandpaper")`
+  describes how templating, translations, and lesson metadata flows from
+  {sandpaper} to {varnish} (added: @zkamvar, #553)
 
 ## BUG FIX
 

vignettes/data-flow.Rmd

@@ -404,10 +404,83 @@ updated with local data (e.g. the sidebar needs to include headings for the
 current page). 
 
 ```{r globals}
+snd <- asNamespace("sandpaper")
 snd$instructor_globals$get()
 snd$learner_globals$get()
 ```
 
+### Translations
+
+In the majority of the {varnish} templates are keys that need translations that
+are in the format of `{{ translate.ThingInPascalCase }}`:
+
+```{r overview, results="asis", echo = FALSE, comment = ""}
+writeLines("```html")
+writeLines(readLines(system.file("pkgdown/templates/content-overview.html",
+      package = "varnish")))
+writeLines("```")
+```
+
+These variables are in {sandpaper} and are _expected to exist_. They get
+generated by the `set_language()` function, triggered when `build_site()` is
+run. When this gets triggered a new list item called "translate" is added to the
+instructor and learner globals.
+
+```{r set-language}
+snd <- asNamespace("sandpaper")
+length(snd$instructor_globals$get()[["translate"]])
+snd$set_language("en")
+Sys.getenv("LANGUAGE")
+length(snd$instructor_globals$get()[["translate"]])
+```
+
+These variables are passed directly to {varnish} templates, which eventually get
+processed by {whisker}:
+
+```{r whisker}
+whisker::whisker.render("Edit this page: {{ translate.EditThisPage }}",
+  data = snd$instructor_globals$get()
+)
+```
+
+This is key to building lessons in other languages, regardless of your default
+language. The lesson author sets the `lang:` config key to the two-letter
+language code that the lesson is written in. This gets passed to the
+`set_language()` function, which modifies the translations inside the global 
+data. Importantly, it is bound to the calling environment, so when the original
+calling environment exits, the language switches back to your default language.
+
+```{r set-language-es}
+local({
+  snd <- asNamespace("sandpaper")
+  snd$set_language("es")
+  print(Sys.getenv("LANGUAGE"))
+  whisker::whisker.render("Edit this page: {{ translate.EditThisPage }}",
+    data = snd$instructor_globals$get()
+  )
+})
+Sys.getenv("LANGUAGE")
+```
+
+### Translation Variables
+
+There are `r length(snd$instructor_globals$get()[["translate"]])` translations
+generated by `set_language()` that correspond to the following variables in
+{varnish}:
+
+```{r translation-list, results = "asis", echo = FALSE}
+withr::with_language("en", snd$add_varnish_translations())
+res <- glue::glue("| `translate.{names(tr)}` | `{tr}` |", 
+  tr = snd$instructor_globals$get()[["translate"]])
+writeLines(c("| variable | string |\n| --- | --- |", res))
+```
+
+In addition, there are other translations that are inserted _before_ they get
+to {varnish}, which you can find by [searching for `tr_` in
+sandpaper](https://github.com/search?q=repo%3Acarpentries%2Fsandpaper+tr_%28+language%3AR+path%3A%2F%5ER%5C%2F%2F&type=code). 
+
+Some general ones can be found in the `get_codeblock_translations()`, 
+`get_callout_translations()`, and `get_accordion_translations()`, 
 
 ## pkgdown metadata
 
@@ -422,12 +495,23 @@ writeLines("```")
 
 This file is what allows {pkgdown} to recognise that a website needs to be
 built. These variables are compiled into the `pkg` variable that is passed to
-all downstream files from `build_site()`:
+all downstream files from `build_site()`. The important elements we use are
+
+ - `$lang` stores the language variable
+ - `$src_path` stores the source path of the lesson (the `site/` directory, or
+   the location of the `SANDPAPER_SITE` environment variable).
+ - `$dst_path` stores the destination path to the lesson (`site/docs`).
+   **NOTE:** this particular path _could_ be changed in `build_site()` so that
+   we update the name of the path so that it's less confusing. The docs folder
+   was a mechanism for pkgdown to locally deploy the site without having to
+   affect the package structure (it was also a mechanism to serve GitHub pages
+   in the past).
+ - `$meta` a list of items passed on to {varnish}
 
 ```{r pkgdown}
 snd <- asNamespace("sandpaper")
 pkg <- pkgdown::as_pkgdown(snd$path_site(lsn))
-pkg
+pkg[c("lang", "src_path", "dst_path", "meta")]
 ```
 
 Before being sent to be rendered, it goes through one more transformation, which

vignettes/translations.Rmd

@@ -43,24 +43,41 @@ in English.
 This vignette is of interest to those who wish to update translations or add new
 translations. In this vignette I will provide resources for updating and adding
 new languages, the process by which translation happens, and I will outline
-special syntax used in {sandpaper}. This process will use the [{potools}
-package](https://michaelchirico.github.io/potools/), which requires the [GNU
-gettext system](https://www.gnu.org/software/gettext/).
+special syntax used in {sandpaper}. ## Resources
 
-## Resources
+### Recommended Tools
+
+To provide translations as a non-maintainer, you need only a text editor to
+add translations to the strings present in the `.po` translation files (see
+more about these files in the [Translating in {sandpaper}
+section](#translating-in-sandpaper).
+
+If you would like to compile and test these translations locally, you will
+need the [{potools} package](https://michaelchirico.github.io/potools/), which
+requires the [GNU gettext system](https://www.gnu.org/software/gettext/). As
+mentioned in the [{potools} installation
+documentation](https://michaelchirico.github.io/potools/#installation), if you
+are on macOS, you will likely need to use `brew` to install `gettext` with `brew
+install gettext`, otherwise, it is bundled with RTools (needed for package
+development on Windows) and it is present on most Linux distributions.
+
+### Documentation
 
 The documentation for the {potools} package is a wonderful resource. Use
 `vignette("translators", package = "potools")` to read details about what to
 consider when translating text in a package. Another really good resource is [a
 blog post by Maëlle Salmon](https://masalmon.eu/2023/10/06/potools-mwe/) which
 gives a minimum working example of translating package messages using {potools}.
 
-If you are interested in translating _lesson content_, please consult tools such
-as Joel Nitta's [{dovetail}](https://github.com/joelnitta/dovetail#readme) for
-providing a method for translators to track and deploy translations of
-Carpentries lessons. You can also use rOpenSci's
-[{babeldown}](https://docs.ropensci.org/babeldown/), which uses the DeepL API
-for automated translation that translators can edit afterwards. 
+If you are interested in translating _lesson content_, please consult Joel
+Nitta's [Carpentries translation guide](https://hackmd.io/@joelnitta/SkCSC6ZNT#Guide-for-translators-l10n). 
+
+You can use tools such as Joel Nitta's
+[{dovetail}](https://github.com/joelnitta/dovetail#readme) for providing a
+method for translators to track and deploy translations of Carpentries lessons.
+You can also use rOpenSci's [{babeldown}](https://docs.ropensci.org/babeldown/),
+which uses the DeepL API for automated translation that translators can edit
+afterwards. 
 
 ## Translating in {sandpaper}
 
@@ -69,24 +86,43 @@ has template variables written in mustache templating. These variables define
 visible menu text such as "Key Points" and screen-reader accessible text for
 buttons such as "close menu". 
 
-Translating in {sandpaper}, you will be working with `.po` files that live in 
-[the po/ folder](https://github.com/carpentries/sandpaper/tree/HEAD/po) in the
-source of this package. There is one `.po` file per language translated and the
-syntax looks like this, where the first line shows the file where the
-translation exists, the second line gives the message in English, and the third
-line gives the translation:
+There are `r length(sandpaper::known_languages())` languages that are known
+to {sandpaper}:
+
+```{r echo = FALSE, results = "asis"}
+writeLines(paste("-", sandpaper::known_languages()))
+```
+
+If a language is not known to {sandpaper} and a lesson attempts to use that
+language, it will default back to English (the source language).
+
+### The source files of translations
+
+When you translate in {sandpaper}, you will be working with `.po` files, which
+are text files that live in [the po/
+folder](https://github.com/carpentries/sandpaper/tree/HEAD/po) in the source of
+this package. There is one `.po` file per language translated. The syntax looks
+like this, where the first line shows the file where the translation exists, the
+second line gives the message in English, and the third line gives the
+translation:
 
 ```po
 #: build_404.R:57
 msgid "Page not found"
 msgstr "ページが見つかりません"
 ```
 
+These files are recognised by [several well-established graphical user
+interfaces](https://michaelchirico.github.io/potools/#alternative-software), but
+since they are text files, you can edit them in any text editor or on GitHub
+directly.
+
+### How translations are processed in R
+
 These po files are compiled into binary `.mo` files that are carried with the
 built package on to the user's computer. These files are used by the R function
 `base::gettext()` to translate messages in a specific context. The context for
-{sandpaper} is known as `R-sandpaper`:
-
+{sandpaper} is known as `R-sandpaper`.
 
 ```{r gettext}
 library("withr")