Make more generic for non-Roblox use cases

[mikejohnstn]

I'm using Luau outside of Roblox. My top problems with this (in terms of LSP-related issues) are:

1. Type checking with require. The require that Luau gives us doesn't work outside Roblox. Until this is fixed in Luau, I'm using a custom require. It takes a string that is a script name without its file extension. Sadly this breaks exported types because luau-analyze doesn't know what to do. Since it may take a while for require to be fixed, some interim solution for this would be great.
2. Built-in types. There's no way to inform luau-analyze of my own built-in types (e.g. types I define in C++). zeus mentioned we can modify luau-analyze itself, or in the future there may also be type definition files. But I need a solution in the meantime. :)
3. OO types. There's no documentation for how to type-annotate OO code. I believe it's doable by casting in the right places. I know it'll be fixed one day with the Records feature. But again, in the meantime I just need something that works.
4. Dependencies. Some way to use luau-lsp without installing anything extra like Rojo.
5. Other editors. Ideally some way to get language features outside of just VS Code as well. The Sublime extension zeux showed looked great if it could just address the above issues.

I know some of these may not make sense to solve in luau-lsp, and maybe in some cases I'm just missing something, but thought I'd share in case anyone has suggestions and/or it's helpful to others.

[JohnnyMorganz]

1. Type checking with require. The require that Luau gives us doesn't work outside Roblox. Until this is fixed in Luau, I'm using a custom require. It takes a string that is a script name without its file extension. Sadly this breaks exported types because luau-analyze doesn't know what to do. Since it may take a while for require to be fixed, some interim solution for this would be great.

We can definitely add support for string based requires (I think we should already have basic support for requiring string paths using require("../module"), but I haven't tested it at all yet).

What kind of things do you need support for from a require perspective? Is (relative/absolute) string path requires sufficient for your use case?

2. Built-in types. There's no way to inform luau-analyze of my own built-in types (e.g. types I define in C++). zeus mentioned we can modify luau-analyze itself, or in the future there may also be type definition files. But I need a solution in the meantime. :)

I'm thinking from the LSP perspective to allow users to pass in paths to definitions files which we will load (#29 - maybe also have a general "global" types configuration too). Support is actually already there, it just hasn't been exposed yet. By default we load in a Roblox definition file, so we should also add an option to disable this.

I actually forked luau-analyze with support to specify any definition files in the command line. If you install the binary from releases, you can then run the following (still currently named luau-lsp, maybe should be named more generally):

luau-lsp analyze --defs=/path/to/definitions.d.lua path/to/file.lua

The original tool was made at https://github.com/JohnnyMorganz/luau-analyze-rojo, but it was later merged into this project (to simplify maintenance). The README in that project should provide some more information, but it is also briefly mentioned at https://github.com/JohnnyMorganz/luau-lsp#standalone.
You can skip any references to Rojo/Sourcemaps/Roblox, it should still work without them.

3. OO types. There's no documentation for how to type-annotate OO code. I believe it's doable by casting in the right places. I know it'll be fixed one day with the Records feature. But again, in the meantime I just need something that works.

I think OO types are just a general Luau thing, so any support for this will be necessary from the Luau side. There is this example: https://luau-lang.org/typecheck#typing-idiomatic-oop, not sure if it applies in your scenario too.

4. Dependencies. Some way to use luau-lsp without installing anything extra like Rojo.

The LSP should actually work without Rojo! You might get some warnings right now (we should probably have a config option to disable these), but the LSP should still be usable.

5. Other editors. Ideally some way to get language features outside of just VS Code as well. The Sublime extension zeux showed looked great if it could just address the above issues.

Definitely open to other editors. I only use VSCode so not sure how language client integrations work with something like Sublime, but if anyone's interested, I'm happy to accept and maintain any language client contributions in the editors/ folder. Hopefully the VSCode extension is reasonably comprehensible to see what is needed from the client side, but it was quickly mashed together - also happy to answer any questions about what's necessary. If I have some extra time, I'll try to look into it later.

Overall, I think generic support is definitely possible, just very primitive right now - need to move some things so they are conditionally registered, but the Roblox/Rojo references can probably be ignored right now and the LSP still usable. I was thinking about maybe an overall setting to define what kind of ecosystem is being used, and guard behind that check. Still need to investigate that a bit more.

[mikejohnstn]

Ah, I didn't realize this should "just work" (with some possible warnings) even without Rojo! I installed it just now directly from VS Code's extension manager, and things look promising so far.

What kind of things do you need support for from a require perspective? Is (relative/absolute) string path requires sufficient for your use case?

Just relative would be great. I can confirm that require() seems to work at least when requiring scripts in the same directory (haven't tried anything else yet).

I'm thinking from the LSP perspective to allow users to pass in paths to definitions files which we will load

#29 looks perfect for this, thanks. It'll help a lot. Is there some other way to force luau-lsp to load a definitions file so I don't have to run luau-lsp analyze manually? Or I guess maybe that's what #29 is for, heh.

You can skip any references to Rojo/Sourcemaps/Roblox, it should still work without them.

Confirmed. Suggestion: consider "inverting" the README so it focuses on instructions for using vanilla Luau. And then add a section for anyone using it with Roblox/Rojo/etc.

I think OO types are just a general Luau thing

You're right, this is probably more a discussion about their docs. In case anyone here has suggestions, my issues with the docs you linked are:

1. They gloss over how to use Account from another file, e.g. having to export type and then having to local myAccount: Account.Account which is pretty weird.
2. They don't give any advice for how to use self with types. Actually maybe this could be addressed on your side. Currently you have to:

function Account.deposit(self: Account, amount: number)
...
end

...in order for luau-lsp to work. Ideally we could use Account:deposit(...) syntax instead.

3. They suggest using the typeof(Account.new()) trick but don't say how to deal with members which themselves are custom types, but are initialized to nil, for example:

function Account.new()
    ...
    self.bank = nil -- where bank is type Bank

   -- I believe the answer is to cast like:
   -- self.bank = nil :: Bank.Bank?

    return self
end

Overall, I think generic support is definitely possible, just very primitive right now

Thanks for the reply, super helpful, and I look forward to ongoing improvements in this direction.

[JohnnyMorganz]

Suggestion: consider "inverting" the README so it focuses on instructions for using vanilla Luau. And then add a section for anyone using it with Roblox/Rojo/etc.

Agreed with this, the README does make it sound very Roblox specific/vanilla support isn't available

2. They don't give any advice for how to use self with types. Actually maybe this could be addressed on your side. Currently you have to:

This is unfortunately a Luau constraint right now, which zeux discussed during his stream about self not being inferred easily. Something that records will hopefully resolve. Luau LSP just acts as a language server backend wrapper around Luau, so the whole type inference etc. is coming from Luau itself.

[JohnnyMorganz]

I think everything brought up in this issue is supported now (apart from other editors - created a new issue at #50)

I'm going to close this, but if there is still places where we lack generic support, feel free to re-open or create a new issue