RFC: deriving_inline ppxlib.traverse

[Lupus]

For validation handling I needed to do some transformations on protobuf OCaml type tree (pb_codegen_ocaml_type) - currently that includes a pass that adds suffixes to all types, and another pass which transforms field types to required if it's instrutcted as such by validation rules. Transforming one protobuf OCaml type tree to another allows to reuse the rest of the code for validated types - like for pretty-printers generation etc.

I'm a big fan of ppxlib.traverse PPX, that generates classes with virtual methods that allow you to traverse your whole type hierarchy in a pre-defined manner (i.e. map it, or fold it, or map it with some arbitrary context passed around). It really shines in complex AST processing tasks, for example I've used it to do rewriting of exceptions to something Golang supports (https://github.com/Lupus/ocaml2go/blob/master/lib/exn_rewriter.re).

I know that extra dependencies, especially ppxes are not welcome in OCaml projects 😭 So I first tried to hand-roll mappers for protobuf OCaml type tree by feeding the types one-by-one to ChatGPT. It did the work reasonably well, and I used that for actual validation generator, but in the long run it's a maintenance burden that won't make anyone happy (not to mention potential bugs in ChatGPT-hallucinated implementation).

I've stumbled upon deriving_inline mode which ppxlib supports for all compatible derivers (including my beloved ppxlib.traverse). I've experimented with this, and it looks like it solves the extra dependencies problems as you don't need any build deps - the code is just there, in source files. It's quite large though, but you don't have to read it, I've organized all types in OCaml type tree as a single recursive type hierarchy (ppxlib.traverse won't work otherwise anyways), which gives a nice bonus that generated code starts after the types, and you don't have to scroll down below that point like at all. dune build @lint and dune promote just do the trick for you, but you need to have ppxlib installed for this to actually work. Good thing about ppxlib.traverse generated code is that it's correct and free of human factor.

Aside of validation plugin, other plugins and internal code in compilerlib itself could likely utilize the traverse primitives for better code modularity and boilerplate reduction.

As an illustrative example, here's how one of the passes looks like in my validation generator right now (using ad-hoc type mappers, but they are very close to API generated by ppxlib.traverse):

  (** [validated_suffix_mapper] is adding _validated suffix to all type names
  and references *)
  class ['ctx] validated_suffix_mapper =
    object
      inherit ['ctx] Type_mapper.mapper_with_context as super

      method! map_user_defined_type (ctx : unit) udt =
        let udt = { udt with udt_type_name = udt.udt_type_name ^ validated_suffix } in
        super#map_user_defined_type ctx udt

      method! map_variant (ctx : unit) v =
        let v = { v with v_name = v.v_name ^ validated_suffix } in
        super#map_variant ctx v

      method! map_record (ctx : unit) r =
        let r = { r with r_name = r.r_name ^ validated_suffix } in
        super#map_record ctx r

      method! map_const_variant (ctx : unit) cv =
        let cv = { cv with cv_name = cv.cv_name ^ validated_suffix } in
        super#map_const_variant ctx cv

      method! map_empty_record (_ctx : unit) er =
        { Ot.er_name = er.Ot.er_name ^ validated_suffix }
    end

This mapper is adding a suffix to all types. I don't care about how nested they are, where they are in the tree, etc - this logic is incapsulated in the traversal code, I only define my business logic, which gives clean separation of concerns.

You can see what is required to do this in this commit: Lupus@2d03538

cc @c-cube