[feature] CUE support

[iyefrat]

Is your feature request related to a problem? Please describe.

From #1598 and #1569 (comment) I see that you're planning on migrating the config files from yaml to pkl. We've been using CUE as our source-of-truth configuration format in our monorepo, for moon.yml, package.json, etc. Have you considered CUE before settling on pkl? Is supporting both an option?

Describe the solution you'd like

Using CUE directly to configure moon. I think that it has a strong set of core abstractions behind it, in particular https://cuelang.org/docs/concept/the-logic-of-cue/

Describe alternatives you've considered

I haven't yet looked into pkl deeply, and currently we have a manual build step, but the fact that we generate the moon files themselves does cause some paper cuts with things like caching and package installation

[iyefrat]

See also: Pulumi supports CUE, and essentially anything that compiles to yaml, via a compile field that lets you specify how to generate the yaml
https://www.pulumi.com/blog/extending-pulumi-languages-with-yaml-cue-jsonnet-rust/

[milesj]

I haven't used cue really but the main reasons I'm looking forward to Pkl is:

• It is owned by a large company. Apple in this case.
• It is backed by a schema (kind of like JSON schema).
• It has custom data types (like data sizes, etc).
• It is programmable. Each data type has methods to mutate/operate with.
• The base config file isn't something like YAML or JSON, it's Pkl.
• I just like the syntax. It's kind of a mix of TOML, Terraform, and Gradle.

I don't really plan to support more than 1 config format, so if Pkl is a success, YAML will be removed entirely in v2.

[iyefrat]

What are your reasons for not wanting to do the pulumi approach, and having yaml (or json, etc) as your config file format, and being unopinionated on how your users compile them? That way the templating logic is still outsourced to the compilation system (pkl/cue/jsonnet/etc), but users retain more flexibility on what they need to use.

[milesj]

@iyefrat I have no idea what pulumi is or what it does, but after a brief glance, I don't see how this would work for moon? moon isn't infrastructure, and a lot of pulumi's APIs are infra centric.

It also seems like a maintenance burden on me (when something breaks) if users are configuring things in whatever format they want. Most support requests are based around config issues.

[iyefrat]

Oh sorry i think i was vague with my wording - I meant the approach linked here #1599 (comment), where you support yaml but let users specify how they want to compile to yaml (via pkl for example), and delegate features like inheritance to the compilation step.

on the moon level you only support the yaml, anything inheritance based is out of scope and delegated to pkl/cue/whatever support.

The main reason I'm concerned about this change is that we use cue to in our monorepo to solve the same class of problems you want to solve here: to define variables, allow shared logic across workspace, not have to reinvent inheritance logic, etc. The issue is that if moon moves to only supporting pkl, we're essentially forced to migrate away from moon, or migrate the monorepo from cue to pkl, and we don't want to be forced in to either of those moves for that reason.

[milesj]

where you support yaml but let users specify how they want to compile to yaml (via pkl for example)

Pkl wouldn't be used to compile to YAML, it would just be Pkl itself. I'm looking for a new configuration format that is stand-alone, and doesn't have a compilation step or intermediary format, but has far more functionality than static formats (JSON, YAML). Something like Starlark is too much, so Pkl feels like a good choice at the moment. I'm not 100% sold on it though, it's still experimental.

The main reason I'm concerned about this change is that we use cue to in our monorepo to solve the same class of problems you want to solve here: to define variables, allow shared logic across workspace, not have to reinvent inheritance logic, etc.

It looks like CUE doesn't even support Rust, so that means it's immediately off the table.

[sharpepd]

@milesj I am trying to find a feature or discussion on the topic of pkl/configuration language, where you can share your thoughts on this topic and get feedback on the experimental pkl feature. I can see a lot of potential in using Pkl, but for our users, it is "yet another language" to learn. The starlark language looks more attractive to use, because of the familiar syntax to Python developers (and we have a number of things already that are python based). So, it would be interesting to have a place where we can see your reasoning behind statements like "starlark is too much".