Rewrite the code generation pass

[hildjj]

Perhaps using a template engine like pug? I've used pug for relatively complex recursive generation in the past, and it worked better than expected.

[hildjj]

The downside of using a template library is that we'd take a runtime dependency. :(

[phpnode]

I've been thinking about this a little bit. What I'd really like is to use a real AST and then we can use @babel/generator and @babel/template but it requires some big changes, and it would also have a huge impact on the file size of the compiler (but not the compiled output).

I suspect a few people are compiling their grammars on the fly via a require hook, so this change might cause some pain for them, but it'd give us a lot more safety and access to a relatively nice toolchain.

If we decide we want to stick with strings then we should probably write a simple template engine ourselves to avoid the pug dependency.

[hildjj]

+1 to a real AST, regardless of what else we do. I also want to be able to generate WASM, btw.

[phpnode]

Do you think we should switch to a plugin model for backends so that javascript should become just one possible output format (albeit the default one)? Btw I just grabbed the @peggyjs org on npm (and invited you, unfortunately @peggy was already taken but at least this mirrors the github org). This way we could have @peggyjs/transform-javascript, @peggyjs/transform-assemblyscript etc

[hildjj]

yes, js should just be a plugin that happens to be included. ts could be first-class at that point, and we could agree with the current maintainer of that plugin to move it into the project if they want. i don't think we're going to be able to maintain backward-compatibility for existing plugins, however.

[reverofevil]

Actually I did it in my rewrite of pegjs, and @babel/types is the way. Templating doesn't help much avoiding repetitive } else { result = FAILURE } code, it's better done with an explicit CFG representation of generated code.

Compiling grammars in browser could be done in an alternative way, by transforming constructions like a = b "+" c into const a = $.seq(b, $.str("+"), c), and supplying a small $ runtime. Generated parser is slower and significantly smaller.