rendering R 4.1's native pipe

[edzer]

I'm not sure whether this is an issue with bookdown, knitr or some other package, but it looks as if the R 4.1 native pipe symbol |> is not rendered correctly:

The same might be the case for the new => infix function operator and the prefix \(

[cderv]

Hi @edzer,

Just to understand correctly, so that we are on the same page: How do you expect |> to look like in the code chunk output of an HTML document ?

Is it because > is highlighted in red ? If so, I believe this would be an issue with the highlighting parser.
Which highlighting configuration are you using ? The default one in gitbook() ? I can reproduce this red color with the default - it is using Pandoc to highlight code chunk and using the pygments theme.

This can be reproduce in a simple document with html_document

---
title: test
output: 
  html_document: 
    highlight: pygments
---

```{r}
f <- function(x) { x + 1 }
2 |> f()
```

```{r}
(\(x) x + 1)(2)
```

I don't know if this is a syntax highlight theme issue or if this requires some changes in the Pandoc syntax highlighter. For gitbook() format, only Pandoc syntax highlighting can be use for now.

Or are you expecting ligatures ? If so I believe for that, you would need to use a font for code that supports ligature. It is the same in Terminal or maybe even in IDE if you use a font that does not support ligatures.

For example, now that rmarkdown integrates with bslib, you could easily change fonts this way. For example with Fira Code, this would look different

---
title: test
output: 
  html_document:
    theme:
      version: 4
      code_font:
        google: Fira Code
---

```{r}
f <- function(x) { x + 1 }
2 |> f()
```

Does it looks like more what you expect ?

BTW There are several ways to change fonts and using bslib with rmarkdown is not the only option.

[cderv]

Note for self:
Looking at the HTML code generated, the Pandoc highlighting gives us this:

<pre class="sourceCode r"><code class="sourceCode r">
<span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a>f <span class="ot">&lt;-</span><span class="cf">function</span>(x) { x <span class="sc">+</span> <span class="dv">1</span> }</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a><span class="dv">2</span> <span class="sc">|</span><span class="er">&gt;</span> <span class="fu">f</span>()</span>
</code></pre>

So |> gives <span class="sc">|</span><span class="er">&gt;</span> with two different classes.

Currently, themes gives different color to those two classes.

Supporting downlit as R parser instead of Pandoc could help solves this - which is something we are looking into: rstudio/rmarkdown#1881

[edzer]

Thanks @cderv for looking into this! I'm looking at the first case - despite the coolness of these ligatures, I think that code sections are most helpful if they show how you would type them. Using this R code:

Sys.setenv("_R_USE_PIPEBIND_"="true") # needs to have it's own code chunk!
f <- function(x) { x + 1 }
2 |> x => sin(x)
2 |> (\(x) { sin(x) })()

using this in your pygments example above, I see

which shows |> and => in two colors, suggesting they're both not a single operator, which they are. That is the issue I have.

[cderv]

Thanks a lot for the example. Then this is indeed due the syntax highlighter used. With gitbook(), Pandoc highlighting will be used and it does not now about this operator. This is a Pandoc issue really for this part. We need to teach the syntax parser in Pandoc about the new operator and / or adapt the highlighting themes.

There would be workaround that have less different colors as mentioned earlier but they are not yet available easily in gitbook().

This is something will definitely need to improve. Thanks for the report on this. I had not tried it yet so I did not know about this issue. 😅

[yihui]

We need to file an issue to Pandoc's syntax highlighter. Or if you are able to, send a pull request (which is probably not too hard): https://github.com/jgm/skylighting/blob/master/skylighting-core/xml/r.xml

Before it is fixed upstream, we can definitely also fix it on our side for now (which should not be hard).

[cderv]

yes I started looking and asked for confirmation on how to do it.

But the changes has to be done upstream in KDE (https://invent.kde.org/frameworks/syntax-highlighting) and not directly in skylighting.

Also --syntax-definition pandoc flag can be useful if we want to import the changes in previous Pandoc version.

[yihui]

This is (temporarily) fixed in rmarkdown. You can test the development version via

remotes::install_github('rstudio/rmarkdown')

Thanks!

[cderv]

Thanks @yihui ! I'll work on the --syntax-definition file for rmarkdown soon so that we can have a proper fix, and push something on Pandoc for later version. I'll open an issue on rmarkdown side for this.

[eddelbuettel]

There is a similar issue in pdf output where

data |> select(colA, colB) |> head()

turns into

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{data }\SpecialCharTok{|}\ErrorTok{\textgreater{}} \FunctionTok{select}\NormalTok{(colA, colB) }\SpecialCharTok{|}\ErrorTok{\textgreater{}} \FunctionTok{head}\NormalTok{()}
\end{Highlighting}
\end{Shaded}

Is there a known trick to get a ligature in pdf?

[cderv]

Hi @eddelbuettel,

The fix in bookdown was only for HTML output. We need to fix it more globally and upstream - this is tracked in rstudio/rmarkdown#2196.

Is there a known trick to get a ligature in pdf?

First thing is the correctly syntax highlighting. The known trick to get correct syntax highlighting would be to use a patch KDE XML syntax definition file.
As I said, we still need to PR upstream in KDE with this fix so that it can go inside Pandoc, then we can benefit.

In the meantime I published a gist with a patched xml file you could use: https://gist.github.com/cderv/079b78caefbe45da6d1c03d737970b55

You could download it and use it by adding pandoc_args in your YAML header. e.g with pdf_document()

output: 
  pdf_document:
    pandoc_args: "--syntax-definition=r.xml"

This would lead to

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{mtcars }\SpecialCharTok{|\textgreater{}} \FunctionTok{head}\NormalTok{()}
\end{Highlighting}
\end{Shaded}

Including this definition file in rmarkdown directly is part of rstudio/rmarkdown#2196 - Still on the todo list though.

Second thing is the Ligature. To get the ligature in the pdf output, I guess you would need a font that supports it like Fira Code.

Here is an example :

---
title: "Minimal Example"
output: 
  pdf_document:
    latex_engine: lualatex
    highlight: zenburn
    pandoc_args: "--syntax-definition=r.xml"
monofont: 'Fira Code'
---

```{r}
mtcars |> head()
```

[eddelbuettel]

Very nice! The syntax checker is already a winner, I am driving this (as my usual slide desks) via binb's variant of metropolis and while it has a monofont option this doesn't (yet) pass through -- but for the one lecture I am using this for it may also better to not just yet jump to ligatures. But I will try to work out where/if the setup gets in the way and overrides.

[cderv]

@eddelbuettel this should now been fixed for pdf_document() using a patch that @yihui pushed. Same trick as for html_document() to wait for upstream support of the native pipe.

This is in the dev version of rmarkdown.

[eddelbuettel]

@cderv Sweet. Given that I have a commonality via my binb package I had contemplated putting the modified XML in there; now I don't need to do that.

The ligature fonts do not come out in pdf if I use binb/metropolis. I pokes a little around the created tex file, and as it goes with such files there are quite a few other packages involved. For now I will keep it at the improved fixed width / non-ligature display in pdf.

Thanks again!

[github-actions]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.