[WIP] Investigate GraphQL for JSON introspection/filtering

[marler8997]

DISCLAIMER: Note that supporting the full GraphQL spec would surely be overkill for this application. This PR is just exploring whether we could leverage a subset of GraphQL to take advantage of the solutions it provides for what we need. Also note that I'm not convinced myself whether this is a good fit for dmd, but I thought seeing if we could leverage the latest technology was worth looking into.

Seeing the new -probe flag being proposed (#7521) and the new data being added to the JSON file (#7757 (review)), I noticed that these use cases and more were just a subset of what you get from using graphql (http://graphql.org/). I decided to look into using a "subset" of graphql for both "filtering json output" and also supporting "introspection" on the json format. For those familair with graphql, I only plan on supporting SelectionSets with Fields (no fragments) (see http://facebook.github.io/graphql/October2016/#sec-Language.Operations). The advantage of graphql is that it will keep the JSON format specification in sync with the JSON that is generated, this being a good application of the DRY principle (see http://graphql.org/learn/schema and http://graphql.org/learn/execution for more info). Here are some examples of it's use:

dmd -Xg=compilerInfo

Note: Assume -Xg=<graphql>
This would output:

{
  "data" : {
    "compilerInfo" : {
      "binary" : "dmd",
      "version" : "2.068.1"
      "config": null,
      "predefinedVersions": [
        "DigitalMars",
        "Posix",
        ..
      ]
    }
  }
}

Here's an example of deeper filtering:

dmd -Xg=compilerInfo{version}

{
  "data" : {
    "compilerInfo" : {
      "version" : "2.068.1"
    }
  }
}

Note that for backwards compatibility, if no graphql query is given, it will default to the original format , namely, an array of objects representing the module files. However, if you want to select that data in graphql, you could do:

dmd -Xg=modules

and get

{
  "data" : {
    "modules" : [
       ...
    ]
  }
}

The filtering is nice, but the main benefit of using graphql will be the introspection. You will be able to use it to query the JSON type system, to know what fields are available. I still need to learn a bit more about GraphQL to provide a good example, so I'll try to update this as I learn a bit more.

[dlang-bot]

Thanks for your pull request, @marler8997! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.

Some tips to help speed things up:

• smaller, focused PRs are easier to review than big ones

• try not to mix up refactoring or style changes with bug fixes or feature enhancements

• provide helpful commit messages explaining the rationale behind each change

Bear in mind that large or tricky changes may require multiple rounds of review and revision.

Please see CONTRIBUTING.md for more information.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

[wilzbach]

That looks like over-engineering and re-inventing the wheel. I would generally recommend to ask for feedback on the NG or Slack before you start such big projects like this one.

Except for the -probe use case (where low overhead is wanted), what's wrong with:

dmd -Xf- | jq .modules

assuming we teach dmd to output the JSON on stdout if requested.

OTOH I have to admit that I like dmd -Xg=compilerInfo ;-)

[marler8997]

I would generally recommend to ask for feedback on the NG or Slack before you start such big projects like this one.

Well I wanted to see if I could convince myself first :) The best way I saw to do that was to see what the implementation might look like. This GraphQL stuff is interesting so this is mostly just an exploration for me and I welcome others feedback along the way.

That looks like over-engineering and re-inventing the wheel.

Well, I was kinda seeing if we could prevent "re-inventing" the wheel by leveraging the solutions that other engineers have already found to these problems. GraphQL gives complete freedom to any application to not only query the exact data it needs, but also to introspect on the data. With a flag like -probe, the responsibility is on the compiler to provide all the applicable information, but also has to find a balance between too little or too much. Usually this ends up causing the compiler to invent interfaces to allow applications to enable/disable subsets of that data. With GraphQL, the interface is defined from the get-go, it never changes, and it supports everything. It might be overkill...I'm not sure.

P.S. I was already planning on developing a GraphQL library, I was just using DMD as a guinea pig user of it. It helps to have one real-world application when developing a library.

[timotheecour]

maybe add an example with field1 and field2?

[timotheecour]

looks like superseeded by #7838 ?

[marler8997]

This is an optional enhancement of #7838, though, I'm leaning on the side of this being "more trouble than it's worth".