Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build stdlibs with rewatch or bsb and integrate Core #6826

Closed
7 of 8 tasks
cknitt opened this issue Jun 23, 2024 · 9 comments
Closed
7 of 8 tasks

Build stdlibs with rewatch or bsb and integrate Core #6826

cknitt opened this issue Jun 23, 2024 · 9 comments
Assignees
Milestone

Comments

@cknitt
Copy link
Member

cknitt commented Jun 23, 2024

Before integrating Core into the compiler repo, we should sort out the standard library build procedure. We should get rid of the ninja scripts and build everything with rewatch (or maybe even still bsb at first).

It turns out to be difficult to set this up because of (among other things) cyclic dependencies that need to be resolved. A lot things need to be shuffled around. It would be easier to iterate on this outside of the compiler repo IMHO.

My proposal is therefore to create a separate repo rescript-stdlib to work on a monorepo structure where all standard libs can be built with a single build command and without relying on any pre-existing artifacts in lib/ocaml. For this, we do not even need the v12 compiler, it should work fine with v11, too. This rescript-stdlib repo would be a temporary one, allowing us to iterate quickly on the standard library build infrastructure. Once we everything is working, it can be taken over into the compiler repo.

A rough plan could look like this:

  • 1. Re-add the -nostdlib option, see Re-add -nostdlib option and make it actually do something #6824.
  • Creeate separate repo for the work on the standard libraries
  • Add a "stdlib mini" (the absolutely essential stuff to get the rest to build), runtime (caml_*.res) and belt and get everything to compile with "-nostdlib -nopervasives" in curried mode
  • Switch to uncurried mode and get runtime to compile
  • Add Core
  • Add (part of?) the old Js namespace as a compatibility package
  • Add (part of?) the old OCaml 4.06 stdlib as a compatibility package
  • Clean up
@cknitt
Copy link
Member Author

cknitt commented Jun 23, 2024

I think the resulting standard library packages could then even be published as separate npm packages (see #6183) independent of a v12 compiler release. They could be used/tested in any ReScript project that sets the -nostdlib compiler option and adds them as dependencies.

@cknitt
Copy link
Member Author

cknitt commented Jun 23, 2024

@zth @cristianoc @jfrolich @rolandpeelen what do you think?

@jfrolich
Copy link

I think it's probably easier to get everything compiled the same way using rewatch in the compiler repo, and refactor from there. This allows us to have the two options side-by-side. Feels like the above approach takes too many steps at once. Sorry I didn't get to finalizing the PR where we compile the stdlib with rewatch but I don't have the impression that it's hard. We just have to determine if we keep the cyclic deps, or refactor them out, we went for the last and that seemed to work. Just need to spend some time to make sure everything is properly compiling.

@cristianoc
Copy link
Collaborator

cristianoc commented Jun 26, 2024

I think the stdlib mini would be pretty nice to have, as it isolates all the special cases. If I understand correctly, everything else can be built user-level on top of it without tricks/surprises. So it seems doable and independent from the details of the other steps.

@cknitt
Copy link
Member Author

cknitt commented Jun 26, 2024

Feels like the above approach takes too many steps at once.

Thanks for the feedback. My feeling was the opposite, that doing it in the compiler would mean taking too many steps at once. IMHO it is hard to refactor all this until it has the proper structure while still keeping the old build process working. (Unless we copy everything to a new dir in the compiler repo, but then we may as well put it into a separate repo temporarily to be able to iterate quickly, like we did for Core itself.)

I think the stdlib mini would be pretty nice to have, as it isolates all the special cases. If I understand correctly, everything else can be built user-level on top of it without tricks/surprises. So it seems doable and independent from the details of the other steps.

Definitely! Everything can be built from scratch in user level, starting with the stdlib mini. (Even with bsb, and even with the v11 compiler with -nostdlib). I'll put up a repo tomorrow with what I have now (basically step 1-4).

@cknitt
Copy link
Member Author

cknitt commented Jun 27, 2024

Ok, so here is my repo: https://github.com/cknitt/rescript-stdlib

It uses compiler v11.1.3-rc.1 with -nostdlib -nopervasives. This ensures that everything is indeed built from scratch.

You can clone it and run npx rewatch to build with rewatch, or npx rescript to build with bsb.

It will then build in one go:

  • a mini stdlib (not quite mini enough yet, I had to add some part of Js, too)
  • belt, depending only on the mini stdlib
  • runtime (caml_*), depending only on the mini stdlib

This is based on #6772, but further modified to get it to build in this setup.

Next up would be adding Core.

@cknitt
Copy link
Member Author

cknitt commented Jun 28, 2024

Moved my repo to https://github.com/rescript-lang/experimental-rescript-stdlib-build

I have also added Core in the meantime and adapted it a bit to fit into the structure.

@cknitt
Copy link
Member Author

cknitt commented Jun 28, 2024

Some questions:

  1. I want to be able to build runtime, belt, Js, and the OCaml stdlib without relying on Core being present (and am already doing so now for the first two). A consequence of this is that common type definitions cannot have Core as their source. I have collected these type definitions in https://github.com/rescript-lang/experimental-rescript-stdlib-build/blob/main/stdlib-mini/src/js_types.res for now. Maybe at least some of these should actually be builtin types?
  2. What we need to be able to rely on though when compiling ReScript code is runtime (caml_*) to be present, see Compiler should not rely on Js or Belt module being present #6836. Therefore the question is, shouldn't the basic types and the "stdlib mini" actually be part of the runtime as caml_types and caml_pervasives or something like that?

What do you think @cristianoc @zth?

@cknitt cknitt added this to the v12 milestone Aug 25, 2024
@cknitt cknitt self-assigned this Aug 25, 2024
@cknitt cknitt changed the title Proposed plan for building all stdlibs with rewatch and integrating Core Build stdlibs with rewatch or bsb and integrate Core Aug 29, 2024
@cknitt
Copy link
Member Author

cknitt commented Oct 22, 2024

Done in #7108 and #7116 (with a simpler approach, no monorepo like in https://github.com/rescript-lang/experimental-rescript-stdlib-build).

@cknitt cknitt closed this as completed Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

3 participants