Migrate provider to terraform-plugin-framework

[mraerino]

The terraform-plugin-sdk used in this provider has some limitations when it comes to newer provider features.

I think it could be beneficial to the project to eventually switch to the newer terraform-plugin-framework.

An example of something that's not possible with terraform-plugin-sdk is nested schemas (issue), which would help a lot with non-string values for custom fields (#349) and general ergonomics of custom fields.

There seems to be a way to use tf5muxserver to gradually migrate a provider to the plugin framework.

[greatman]

I've started to play around migrating the provider to v6 / terraform-plugin-framework & the new test framework (terraform-plugin-testing): https://github.com/greatman/terraform-provider-netbox/tree/plugin-framework

So far, seems possible rather easily through the mux server feature like suggested. I have a v6 protocol provider using plugin-framework and a v5 protocol with the old SDKv2.

It does require rewriting about everything but at least the mux server allows to migrate 1 piece at a time. The tests runs through the mux server so they still have access to everything. All tests except the old netbox_site v5 tests works.

The new main implementation:
https://github.com/greatman/terraform-provider-netbox/blob/plugin-framework/main.go
https://github.com/greatman/terraform-provider-netbox/blob/plugin-framework/netbox/provider_plugin.go
https://github.com/greatman/terraform-provider-netbox/blob/plugin-framework/netbox/resource_netbox_site_v6.go
https://github.com/greatman/terraform-provider-netbox/blob/plugin-framework/netbox/resource_netbox_site_v6_test.go (Last test uses all new features of the test framework)

Some other modifications around. First project i'm working on Go ever, so all new stuff for me 😛

[fbreckle]

If I had more time, I would take an in-depth look at re-generating this provider mostly from scratch with https://www.hashicorp.com/blog/terraform-provider-code-generation-now-in-tech-preview (which would be using the framework then)

I think with these tools plus some glue code, most of the code base could be auto-generated.

[mraerino]

yeah, auto-generation would be amazing, but i feel like there are a couple nasty edge cases.

would it make sense to maybe do a virtual hackday to give this a spin or something like that?

[greatman]

I explored yesterday the code gen module of Terraform, that's what I discovered:

• It allows to generate scaffolding for whatever you want. Cuts a bit on boiler plate.
• From a openAPI 3.0 spec YML file, it allows you to generate Resources & Data Schemas with validations. I had to edit a bit the yml to be fully compatible with https://github.com/netbox-community/go-netbox/blob/master/scripts/fix-spec.py and some manual edits.
• You still need to craft most of the logic to link terraform with the API.

Some example of what scaffolding it generates:

• Resource: https://github.com/greatman/terraform-provider-netbox-generated/blob/master/internal/provider/cluster_resource.go
• Datasource: https://github.com/greatman/terraform-provider-netbox-generated/blob/master/internal/provider/site_data_source.go
• Schema it generates:
  https://github.com/greatman/terraform-provider-netbox-generated/blob/master/internal/resource_site/site_resource_gen.go

To generate the _gen files, the following was used:
tfplugingen-openapi generate --config api/provider_config.yml --output api/provider_code_spec.json api/openapi.yaml
tfplugingen-framework generate all --input api/provider_code_spec.json --output internal

To generate a base scaffolding, the following: tfplugingen-framework scaffold resource --name cluster --output-dir ./internal/provider

So from a quick look, good to increase velocity on heavy boilerplate like generating Schemas & validation, but still need to write all the logic.

With both the mux server & that, the conversion could be faster.

[fbreckle]

Yes, but like 80% of the logic of a terraform provider is mapping the API attributes to their respective hcl attributes via a go library (in this case go-netbox). These fields are usually just strings or number, lists of these, or pointers on these.

There obviously remain some edge cases where custom behavior is needed like custom fields, tags, the available_* resources and the provider configuration itself (and probably more), so that has to be considered.