Vignette classes for syntax highlighting are mixed up

[dusadrian]

The Vignette syntax highlighting does not seem to follow the general structure of the html classes.

Something like aa <- c(1, 2) is rendered in the html version as

aa &lt;-<span class="st"> </span><span class="kw">c</span>(<span class="dv">1</span>, <span class="dv">2</span>)

I believe the function c() should in fact have a class "fu" (not keyword "kw"). Similarly, something like runif(n = 3) is rendered as:

<span class="kw">runif</span>(<span class="dt">n =</span> <span class="dv">3</span>)

where the argument n should have a different class from DataType ("dt"), perhaps "at" if I am not mistaken. Is there a special reason why these classes are mixed up?

Below is the output from:

> xfun::session_info('rmarkdown')
R version 4.2.0 RC (2022-04-15 r82193)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Monterey 12.4

Locale: en_US.UTF-8 / en_US.UTF-8 / en_US.UTF-8 / C / en_US.UTF-8 / en_US.UTF-8

Package version:
  base64enc_0.1.3  bslib_0.4.0      cachem_1.0.6     digest_0.6.29    evaluate_0.16    fastmap_1.1.0   
  fs_1.5.2         glue_1.6.2       graphics_4.2.0   grDevices_4.2.0  highr_0.9        htmltools_0.5.3 
  jquerylib_0.1.4  jsonlite_1.8.0   knitr_1.39.7     magrittr_2.0.3   memoise_2.0.1    methods_4.2.0   
  R6_2.5.1         rappdirs_0.3.3   rlang_1.0.4      rmarkdown_2.15.1 sass_0.4.2       stats_4.2.0     
  stringi_1.7.8    stringr_1.4.0    tinytex_0.41     tools_4.2.0      utils_4.2.0      xfun_0.32       
  yaml_2.3.5      

Pandoc version: 2.9.2

Checklist

When filing a bug report, please check the boxes below to confirm that you have provided us with the information we need. Have you:

• formatted your issue so it is easier for us to read?

• included a minimal, self-contained, and reproducible example?

• pasted the output from xfun::session_info('rmarkdown') in your issue?

• upgraded all your packages to their latest versions (including your versions of R, the RStudio IDE, and relevant R packages)?

• installed and tested your bug with the development version of the rmarkdown package using remotes::install_github("rstudio/rmarkdown")?

[yihui]

I tested the latest version of RStudio, which bundles Pandoc 2.18, and it worked as you expected.

> xfun::session_info('rmarkdown')
R version 4.2.1 (2022-06-23)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Monterey 12.5, RStudio 2022.7.1.554

Locale: en_US.UTF-8 / en_US.UTF-8 / en_US.UTF-8 / C / en_US.UTF-8 / en_US.UTF-8

Package version:
  base64enc_0.1.3 bslib_0.4.0     cachem_1.0.6    digest_0.6.29   evaluate_0.16   fastmap_1.1.0  
  fs_1.5.2        glue_1.6.2      graphics_4.2.1  grDevices_4.2.1 highr_0.9       htmltools_0.5.3
  jquerylib_0.1.4 jsonlite_1.8.0  knitr_1.39.7    magrittr_2.0.3  memoise_2.0.1   methods_4.2.1  
  R6_2.5.1        rappdirs_0.3.3  rlang_1.0.4     rmarkdown_2.15  sass_0.4.2      stats_4.2.1    
  stringi_1.7.8   stringr_1.4.0   tinytex_0.41    tools_4.2.1     utils_4.2.1     xfun_0.32      
  yaml_2.3.5     

Pandoc version: 2.18

So if you use RStudio, you may update RStudio, otherwise update Pandoc.

[dusadrian]

That worked indeed, it was Pandoc's version that mixed things up.
Thanks a lot.

[dusadrian]

Actually, there might be one more thing to add, here.
The classes now work fine when highlighting in code chunks, but not in regular inline code. Below a reprex:

Demonstrating the `runif()`{.r} function:

```r
runif(n = 3)
```

That renders as:

<p>Demonstrating the <code class="sourceCode r"><span class="fu">runif</span>()</code> function:</p>
<div class="sourceCode" id="cb1"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="fu">runif</span>(<span class="at">n =</span> <span class="dv">3</span>)</span></code></pre></div>

This is a screenshot of how it actually looks:

So the two functions are highlighted differently, despite having the same class "fu". Presumably, this is due to lines 101 and 326 of the resulting html of the vignette:

code span.fu { color: #06287e; } 
code > span.fu { color: #bb3730; font-weight: bold; } 

[yihui]

You are correct. I don't know why Pandoc defines CSS rules differently for code > span.fu and code span.fu. You can redefine code > span.fu (e.g., in a css code chunk). Anyway, I think this is not directly relevant to R Markdown.

[dusadrian]

Indeed, I just hoped this can be solved somehow by rmarkdown, since the final result is an html file irrespective of Pandoc (perhaps post-processing the html file to re-write those classes, at least this is what I would do).

[cderv]

Just a precision as I explained in #2360 (comment) regarding the styling,

I think the rules in the CSS here

rmarkdown/inst/rmarkdown/templates/html_vignette/resources/vignette.css

Lines 190 to 201 in 45c6802

	code > span.kw { color: #555; font-weight: bold; } /* Keyword */
	code > span.dt { color: #902000; } /* DataType */
	code > span.dv { color: #40a070; } /* DecVal (decimal values) */
	code > span.bn { color: #d14; } /* BaseN */
	code > span.fl { color: #d14; } /* Float */
	code > span.ch { color: #d14; } /* Char */
	code > span.st { color: #d14; } /* String */
	code > span.co { color: #888888; font-style: italic; } /* Comment */
	code > span.ot { color: #007020; } /* OtherToken */
	code > span.al { color: #ff0000; font-weight: bold; } /* AlertToken */
	code > span.fu { color: #900; font-weight: bold; } /* Function calls */
	code > span.er { color: #a61717; background-color: #e3d2d2; } /* ErrorTok */

are no more used, or at least should not be used or just should be updated.

html_vignette uses Pandoc highlighting by default - so highlighting style is controlled by Pandoc's style, which can be changed in highlight parameter, including using a custom style file I believe. (using Pandoc format).

I am not sure in which context the rules of vignettes.css have been added, but if they conflict then they should probably be removed. 🤔

Regarding the parsing of the R Code for HTML highlight, this is done by Pandoc also. Pandoc version update can also update the parser. This is why updating the version of Pandoc may have fix the class issue if it was fixed upstream. If there are still some incorrect highlighting, this is something to report upstream in KDE which is where the highlighting parser comes from.

since the final result is an html file irrespective of Pandoc (perhaps post-processing the html file to re-write those classes, at least this is what I would do).

@dusadrian which part is irrespective of Pandoc ? The CSS rules right ? or another part.

thanks

[yihui]

but if they conflict then they should probably be removed

I tend to agree. I didn't realize we had a stylesheet vignette.css by ourselves. Thanks for the explanation!

[dusadrian]

Oh, that explains a lot...
By contrary, I had assumed vignette.css was there on purpose and played a central role, and never crossed my mind this is a forgotten relic of some sort.

@cderv, yes indeed, I was referring to the CSS rules. If nothing else worked, there is at least the option of post-processing the resulting html file(s) to modify, delete, or add rules to obtain the desired result. But I would avoid this, if at all possible.

For the moment, I managed to solve this by employing a custom.css style, as suggested by @yihui and advised in the manuals. I simply copied vignette.css and modified the colors from those CSS classes, seems to work.

PS: this is a link to the place where I used this in practice, if it helps:
https://github.com/dusadrian/declared/tree/main/vignettes

[cderv]

@dusadrian the official mechanism to use a custom style for highlighting is to provide a custom them in highlight argument.
See example of such file in the package itself where we provide a custom arrow.theme https://github.com/rstudio/rmarkdown/blob/main/inst/rmarkdown/highlight/arrow.theme

The default one used in Pandoc for our vignette format can be obtained using

pandoc --print-highlight-style pygments

See https://pandoc.org/MANUAL.html#option--highlight-style

This allows to tweak the default one, if no other option suits you.

Benefit of doing that is to have a theme that works with PDF and DOCX too, and any other format supported by Panodc highlighting.

The simple method of overwritting Pandoc's included CSS rules for the HTML is the other method. You are doing that and that is perfectly fine if you are using HTML output only for this style !

Hope this clarifies. We'll probably remove the rules in the vignette.css file - I just need to check if some of them are used somehow - from your report I am wondering. If we just remove we could end up with users asking why it has changed 😅

[dusadrian]

Indeed, and I have previously, already tried the Pandoc theme way (banging my head to no avail). Copying your arrow.theme into a local file declared.theme, it works as expected for code chunks but still not for inline code:

I am now struggling to understand where and how does Pandoc use that class for user_na_to_na() (which is my own custom class), since I have replaced the css: "declared.css" with highlight: "declared.theme" in the YAML header...?
(I've even physically moved that file from the vignettes directory and Pandoc still uses it).

title: "Motivation for the declared package"
output:
  rmarkdown::html_vignette:
    highlight: "declared.theme"

[cderv]

it works as expected for code chunks but still not for inline code

Oh now I think I missed something in your explanation. I just saw that use add a class to the inline code, right ?

Demonstrating the `runif()`{.r} function:

Are you doing the same thing in the example above with user_na_to_na() ?

I am just discovering that applying a class using this syntax will trigger the syntax highlighting even in the inline code with Pandoc. I forgot about that. Usually, no class is applied and inline code are not highlighted specifically. Thanks for reminded me.

Anyway,

I am now struggling to understand where and how does Pandoc use that class for user_na_to_na() (which is my own custom class),

Not sure to understand here what you mean. Which class are you talking about ?

In your example, as discussed in previous message, you see a red color and not the same color as in code chunk even with applying the {.r} class on the inline code just because of the CSS we are adding in vignette.css and that we will remove for next version.

One way of getting rid of it now is post processing the HTML, or passing a custom css in vignette format so that the default one is not included (could be a copy of the default one without the CSS rules at the end).

At the end, I am not quite sure of the result you are looking for - you can send me your declared.css and declared.theme so that I can reproduce too and see the content.

Sorry everything got mix up for me and I am not sure to see if there is another issue you are trying to mention apart for the CSS in vignette.css that you can control insertion through css argument.

As a reminder:

• If you use css: declared.css in the YAML, the default one won't be added so not specific overwrite rules
• If you use only highlight: declared.theme, vignette.css will be added which contains the conflicting rules that we should remove hence a difference in color.

[dusadrian]

Indeed, I am using the .{r} notation to highlight inline code.

You are absolutely right, that red bold color is the result of the vignette.css file, I was confused because it was so very similar to my own highlighting (I use #cb2626).

Thanks for all the explanation, everything is alright now. I am going to continue using the declared.css file (essentially a slightly modified vignette.css for some colors), until those classes will be removed from vignette.css. At that point, the mere usage of a .theme file will be enough.