Can MIR be used for Crystal?

[vlazar]

I've started a discussion on Crystal forum https://forum.crystal-lang.org/t/can-crystal-use-mir-for-faster-compiler-interpreter/3851

The answer from one of the core team members seem to be that MIR does not have enough types.

So I wonder if using MIR for Crystal possible/a good idea?

[vnmakarov]

Argument that MIR has no types bigger 64-bit means it can not be used for static languages is wrong.

There is already C JIT based on MIR (c2m). C is a static type language.

Static arrays and structures can be represented by data/bss and pointers to them. Dynamic arrays/structures can be stored in heap or on stack and access by pointers (64-bit value) to them.

What I see people who use LLVM for their language implementation are tired of LLVM API instability, difficulties to support different versions of LLVM, LLVM size and compilation speed, difficulties to get review of their patches for LLVM, etc.

The current actual problem with MIR is that it is oriented to JITs and does not permit to generate object files and as consequence to be easily used for AOT.

[vlazar]

Thank you for detailed answer!

And what is the debugging story with MIR currently? I'm trying to gather more info for Crystal forum post on Crystal interpreter and MIR. This question in particular:

Does MIR support nice debugging?

Upcoming Crystal interpreter is a hand written byte code generator / VM and it supports debugging (REPL).

I've searched through the issue tracker and if I get it correctly debugging is hard currently with MIR and it wasn't clear clear to me if you plan to work on making it easier in the near future. You've mentioned, that supporting DWARF generation is a big project and is unlikely to happen.

Since one of you long term goals is trying MIR JIT for CRuby I would imagine you have to think about debugger support too at some point? I mean if MIR to be picked as the next JIT for Ruby doesn't it mean that high level debugging in IRB should be possible with MIR JIT and work exactly the same as now?

[vnmakarov]

And what is the debugging story with MIR currently? I'm trying to gather more info for Crystal forum post on Crystal interpreter and MIR. This question in particular:

Does MIR support nice debugging?

I had no big troubles with MIR ocde debugging, I used gdb on machine level (stepi, disasm, etc) and a lot of debug information printed by MIR-generator. It was enough for me. But I understand my case is very specific as I am the author of all code.

I understand debugging is very desirable feature for MIR users. I added tracing insns in MIR-interpreter which shows all executed insns and their results.

Currently I don't understand how debugging of MIR-code should look like. Should it be on MIR-level or on C level as people use c2m. What level is the most important right now. I hope my future experience with using MIR for CRuby JIT will give me answers.

Upcoming Crystal interpreter is a hand written byte code generator / VM and it supports debugging (REPL).

I've searched through the issue tracker and if I get it correctly debugging is hard currently with MIR and it wasn't clear clear to me if you plan to work on making it easier in the near future. You've mentioned, that supporting DWARF generation is a big project and is unlikely to happen.

DWARF generation could be used only when object code (elf, macho) is already implemented. Currently, MIR generates only machine code. I don't know when object files will be generated or should I work on it.

The current my goal is CRuby JIT implementation based on MIR. I am at the start of this work and I already see additional things should be added to MIR-project but they do not include object file generation.

Since one of you long term goals is trying MIR JIT for CRuby I would imagine you have to think about debugger support too at some point? I mean if MIR to be picked as the next JIT for Ruby doesn't it mean that high level debugging in IRB should be possible with MIR JIT and work exactly the same as now?

I did not investigate will irb work with JIT. I don't see big obstacles for this.

Currentlyl I have plans to implement debugging using gdb JIT interface https://sourceware.org/gdb/current/onlinedocs/gdb/JIT-Interface.html in the future. I think it will be useful for me and other people who will try to use MIR project for their JITs.

[vlazar]

Thank you for detailed answer. Everything is much more clear now!

[dibyendumajumdar]

The answer from one of the core team members seem to be that MIR does not have enough types.

MIR IR does not have support for aggregate types in the IR - unlike LLVM. This does make it harder to use MIR as you have to code the logic for accessing aggregate data structures yourself. That's why I use the C as the intermediate language in Ravi - since MIR comes with a C compiler that does all this for you.

[vnmakarov]

This does make it harder to use MIR as you have to code the logic for accessing aggregate data structures yourself.

I am absolutly agree. Generating MIR-code accessing structures is not easy as it should be done in machine dependent way with knowledge of layout algorithm which can be quite complicated.

But there is a work around for this. The code describing structures and inline access functions could be written on C. The C code could be translated to MIR by c2m and directly generated MIR-code should use only calls to the functions.

[luikore]

Compiler frontend can implement a custom debugger, by adding debug instruments in their code like YARV's trace instruction. But this way it doesn't play well with GDB or LLDB.

Maybe simple debugging support should be enough for many use cases? In GNU assembler, the basic debug info is .file and .loc. They allow GDB or LLDB to step on the source code level. For JIT the info should can be put in an in-memory DWARF format (https://sourceware.org/gdb/current/onlinedocs/gdb/JIT-Interface.html).

Mapping registers back to variables or adding type info looks more difficult. But we can write debugger extensions to interpret the stack frame so these tasks can somehow be done without MIR.