Add porter schema root command

[carolynvs]

$ ./bin/porter schema
{
    "type": "object",
    "properties": {
      "name": {
        "type": "string"
      },
      "description": {
        "type": "string"
      },
      "version": {
        "type": "string"
      },
      "image": {
        "type": "string"
      },
      "mixins": {
        "type": "array",
        "items": {
          "type": "string"
        }
      }
    },
    "additionalProperties": true
}

Note: This isn't the full json schema. We will fill that in once we validate this these commands are plumb together (between an IDE, porter and the mixins) to help provide autocomplete/intellisense.

I'm following up on this PR with another that adds porter schema --mixin NAME but figured it would be easier to review incrementally. EDIT: After talking with @itowlson this is on hold, may not need a subcommand.

Closes #76

Here's all the things I found when shaving this yak that need to be split out into separate PRs most likely:

• Porter not mixin anymore
• Go generate or packr build or packr clean. We aren't using packr right in our builds. (Fix how we use packr in our builds #193)
• Remaining mixin functions to interface as issue (Isolate mixin functions into separate interface on porter #192)
• Helm and azure schema sync up (this didn't fix the schema to report all the fields, just to have them report the correct shape as per Rearrange step schema #182)

[itowlson]

This looks like a good starting point (noting your remark that this is not the final schema). My main concern is to what extent consumers will want to post-process the schema, e.g. converting the mixins type to an enum instead of free text. But I guess that can be down to the consumer - having an authoritative starting point is surely better than having to write the whole thing from scratch. One thing that might help with this is to move stuff out to a definitions section, so consumers have a clear place to look for definitions, though I don't know whether this would really matter in practice. (I found it helped me get my thoughts straight when I was working with base+mixin schemas in the VS Code POC.) Such a change might be hard to make after release because tools that post-process the schema will depend on its layout.

In any case, thanks heaps for thinking about the tooling. I do feel this is on the right track even if it does take some iteration to get it playing nicely with the mixin scenario.

[carolynvs]

Switching back to WIP. After talking with @itowlson I'm going to have it work this way instead:

$ porter schema
< dump a consolidate / merged JSON schema for the entire porter.yaml taking into consideration the mixins installed on the system>

The other command porter schema --mixin FOO is on hold, may not ever be needed.

[carolynvs]

@itowlson Here is what the command looks like now. Let's ignore the helm mixin definition for now, I'm going to need to go back and fix it, since it's in another repository. I did do the exec mixin since it's in this repo. How well does this match what you expected and what VS Code requires?

The list of definitions is dynamic based on what mixins are installed, and so is the mixins enum, and the refs under install. Once we have install working, I'll do upgrade and uninstall as well.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "additionalProperties": false,
  "definitions": {
    "exec.install": {
      "properties": {
        "description": {
          "type": "string"
        },
        "exec": {
          "properties": {
            "arguments": {
              "items": {
                "type": "string"
              },
              "type": "array"
            },
            "command": {
              "type": "string"
            }
          },
          "type": "object"
        }
      },
      "type": "object"
    },
    "helm.install": {
      "additionalProperties": true,
      "properties": {
        "chart": {
          "type": "string"
        },
        "name": {
          "type": "string"
        }
      },
      "type": "object"
    }
  },
  "properties": {
    "description": {
      "type": "string"
    },
    "install": {
      "anyOf": [
        {
          "$ref": "#/definitions/exec.install"
        },
        {
          "$ref": "#/definitions/helm.install"
        }
      ],
      "type": "array"
    },
    "invocationImage": {
      "type": "string"
    },
    "mixins": {
      "items": {
        "enum": [
          "exec",
          "helm"
        ],
        "type": "string"
      },
      "type": "array"
    },
    "name": {
      "type": "string"
    },
    "version": {
      "type": "string"
    }
  },
  "type": "object"
}

If it's easier than reading the schema, I can give you a binary to try out locally. Just let me know.

[itowlson]

That looks like it's going the right way, but I'd consider refactoring the install definitions so that description is part of install rather than each mixin having to report it. (Unless you want mixins to be able to define properties outside the scope of the <mixin_name>: element of course.)

I think the way I had this in my POC evaluated to something like:

// in the definitions section (the actual schema
// for install would be an array of this)
"install": {
  "type": "object",
  "properties": {
    "description": { ... },
    "outputs": { ... },
    "exec": { "type": "object", "$ref": "#/definitions/exec-install" },
    "helm": { "type": "object", "$ref": "#/definitions/helm-install" },
    "oneOf": [
      { "required": "exec" },
      { "required": "helm" }
    ]
  }
}

This meant that (what I took to be) common stuff like description and outputs would be declared at the step level and it was only the exec: ... or helm: ... bodies that would have to be declared by the mixin. I now realise I did not check my assumptions with you though!

Hope this makes sense - happy to explore in more detail if you want. I'm not trying to mandate what I came up with - in fact I prefer anyOf to the oneOf: [ { required... } ] trick - just trying to keep the mixin implementation side simple if I have understood it right.

[carolynvs]

I just double checked and moving outputs under the mixin would not have any negative effect on porter. Description was being used by porter for logging progress such as "now executing the DESCRIPTION step". But that's ok to move too.

It is tidier to move it all under the mixin. I'll try that, and keep the "anyOf" since you like that better.

Hopefully this is getting closer to an end. Sorry it's taking so long!

[itowlson]

Sounds good - I'll load it up into VS Code and see how it plays out in terms of proferred popups and validation error messages.

[itowlson]

VS Code plays nicely with the proposed output - or at least no worse than it did with any of the other ways I've tried expressing the schema. The main difference I noticed was if I had both a helm and exec in a step: my old schema reported Matches multiple schemas when only one must validate and the new one reports Unexpected property helm. I think the new one is probably more accurate and informative.

I did need to make a few tweaks, but those were mostly relating to the Helm stuff which you hadn't done yet. We also need some required elements to get Code to whinge if e.g. you supply a step description with no actual body! Here's where I ended up in my testing:

  {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "additionalProperties": false,
    "definitions": {
      "exec.install": {
        "properties": {
          "description": {
            "type": "string"
          },
          "exec": {
            "properties": {
              "arguments": {
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "command": {
                "type": "string"
              }
            },
            "type": "object",
            "additionalProperties": false
          }
        },
        "required": ["exec"],
        "type": "object",
        "additionalProperties": false
      },
      "helm.install": {
        "properties": {
          "description": {
            "type": "string"
          },
          "helm": {
            "properties": {
              "chart": {
                  "type": "string"
              },
              "name": {
                  "type": "string"
              }
            },
            "type": "object",
            "additionalProperties": false
          }
        },
        "required": ["helm"],
        "type": "object",
        "additionalProperties": false
      }
    },
    "properties": {
      "description": {
        "type": "string"
      },
      "install": {
        "type": "array",
        "items": {
          "anyOf": [
            {
              "$ref": "#/definitions/exec.install"
            },
            {
              "$ref": "#/definitions/helm.install"
            }
          ]
        }
      },
      "invocationImage": {
        "type": "string"
      },
      "mixins": {
        "items": {
          "enum": [
            "exec",
            "helm"
          ],
          "type": "string"
        },
        "type": "array"
      },
      "name": {
        "type": "string"
      },
      "version": {
        "type": "string"
      }
    },
    "type": "object"
  }

[carolynvs]

@jeremyrickard Rebased for great glory

[carolynvs]

Okee, one more rebase to pick up the go generate PR and resolve some conflicts.

[vdice]

Phew! Awesome work on this. A few questions/comments at this stage for me...

[jeremyrickard]

LGTM. I think the stderr logging should get updated in a follow up PR.

[carolynvs]

Merge like this PR is the albatross that ruined the month of February. 🐦

[carolynvs]

I've created #199 and #200 to track the follow-up work from this PR.