feature: sagetex implementation

[marrsus]

is it possible to add sagetex implementation
i am a math student and i use latex for my assignments i do a lot of all of my calculations in sagetex i would really love to have code completions and highlighting while inside sage blocks i know a bit of rust myself and would be happy to help but someone need to point my in the right direction

[pfoerster]

@marrsus Thanks for the suggestion. To provide code completion, I think the following steps would be required:

1. The first step to add support for sagetex would be adding a Sage parser. I don't find a dedicated LSP server for Sage so we cannot rely on the ideas mentioned in Embedding other LSP servers for code examples in a tex document #824.
2. Link the sagetex fragments to the LaTeX parse tree (language injections) and maybe do some quick analysis.
3. Implement the textDocument/completion request using the parse trees and the analysis results.

Overall, this is a lot of effort. If you just need syntax highlighting, then, this does not need to be done in the server. In fact, the server does not currently provide syntax highlighting yet. The steps depend on the LSP client / editor. If you are using tree-sitter, then this is easiest done by extending tree-sitter-latex grammar and configure the editor to use the language injection feature.

[marrsus]

sounds like this is a little over my level, so i don't know if a can implement it myself, but thanks for the help anyway, and thanks for the tree-sitter tip, that is one less thing to miss.

just for the record sagemath is basically like a library for python so if someone wanted to use an outside LSP server it would likely be a python one

[thelonesomeprogrammer]

i really want this feature as well and i have looked into it.
would i be correct that the first steps to add sagetex parser would be something along the lines of adding a sage command token to the parser and then make a sub parser for the sage code itself

[pfoerster]

@thelonesomeprogrammer

would i be correct that the first steps to add sagetex parser would be something along the lines of adding a sage command token to the parser and then make a sub parser for the sage code itself

Thanks for tackling this!
In short, the following steps are required:

1. Add a parser for Sage snippets (see crates/parser and crates/syntax). Note that the parser needs to be lossless so it should not return errors if the sage code is invalid.
2. Extend the LaTeX parser for locations where Sage snippets can be embedded; the LaTeX parser should not parse the complete snippet here.
3. Collect and parse all sage snippets when walking the LaTeX tree (see crates/base-db/src/semantics/tex.rs)
4. Extend the completion/hover requests

Please let me know, if you need further assistance!

[thelonesomeprogrammer]

@pfoerster for the parser how am i best to add sagetex i see the option to add it in the content function or the begin environment function i figure that i have to take it as the sagetex portion should interfere with the regular latex

[pfoerster]

@thelonesomeprogrammer I would not add it to the environment function. Instead, I would parse it during analysis (see https://github.com/latex-lsp/texlab/blob/master/crates/base-db/src/semantics/tex.rs#L45).

as the sagetex portion should interfere with the regular latex

In this case, it would be better to let the parser treat it as a verbatim environment and just let it search for the \end{...} and then parse the sagetex portion afterwards.

[LeoSchae]

I have not used VSCode in a while, but I believe syntax highlighting for latex is performed by vscode itself and not the language server.

If one wants syntax highlighting to work, the best way would probably be to inject a grammar into the syntax of VSCode. The website has a long guide on syntax. Maybe there is also a guide somewhere that does exactly what you want.
https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide#injection-grammars
The subsection for what we want would be Embedded languages. I know that tree-sitter supports something similar and I wanted to look into it sometime. Sadly, I do not have a lot of free time on my hands right now.

For Autocomplete, there is also a section on how one does this for language servers.
https://code.visualstudio.com/api/language-extensions/embedded-languages

But I believe that this is a lot more work. The syntax highlighting seems quite doable.