-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expose crosstests.Create
#2501
Expose crosstests.Create
#2501
Conversation
This begins to turn cross-tests into a library with a defined interface. Existing cross-tests say that they are a prototype level API, and they feel that way. The prototyping was successful, and cross-tests have proven their worth. We should begin to move towards a more discover-able API. The first function of which is: ```go // crosstests/create.go func Create( t T, resource map[string]*schema.Schema, tfConfig cty.Value, puConfig resource.PropertyMap, options ...CreateOption, ) ``` We can continue to iterate on this API, but we should begin to formalize a boundary between the implementation of cross-tests and tests written with cross-tests. stack-info: PR: #2501, branch: iwahbe/stack/2
99b62b9
to
c75ca97
Compare
9e19976
to
1df47a7
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## iwahbe/stack/1 #2501 +/- ##
==================================================
- Coverage 62.58% 62.50% -0.08%
==================================================
Files 380 381 +1
Lines 51270 51335 +65
==================================================
+ Hits 32087 32088 +1
- Misses 17368 17432 +64
Partials 1815 1815 ☔ View full report in Codecov by Sentry. |
pkg/tests/cross-tests/create.go
Outdated
// | ||
// Create *does not* verify the outputs of the resource, only that the provider witnessed the same inputs. | ||
func Create( | ||
t T, resource map[string]*schema.Schema, tfConfig cty.Value, puConfig resource.PropertyMap, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to expose both the pulumi and TF inputs in the test framework? I don't think it is the common use case for these to differ - should we instead aim to make it simpler to use this by having one input and handle the conversion in the framework, as before?
I don't think we have that many use cases for the inputs differing outside of type conversions as it this defeats the purpose of the test
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I said in #2500 (comment):
We need to test:
- implicit type conversions
- explicit type changes (Testing: Add an integration test for type overrides #2048 is a start)
- max items one fiddlyness (simulating major version upgrades)
- secret injection (if we ever want to handle secrets, or in Configure)
- etc.
Another example: If you want to verify that set order doesn't matter, you would need to do something like:
for _, setPermutation := range permutationsOf(setValues) {
crosstests.Create(t, schema, tfConfig, resource.PropertyMap{
"set": resource.NewArrayProperty(setPermutation),
})
}
I don't think you could reliably do that in tftypes.Value
space.
From Create
's API, it would be very easy to build in automatic pulumi value inference:
crosstests.Create(t, schema, tfConfig, crosstests.InferValue)
Just as easily, we could make accepting the Pulumi value an option, defaulting to inference:
crosstests.Create(t, schema, tfConfig) // Pulumi value is inferred
crosstests.Create(t, schema, tfConfig, // Pulumi value is explicit
crosstests.CreatePulumiValue(explicitPulumiValue))
As long as the internals support explicit values (#2500), it's easy to convert to an implicit value. From there, there are many ways we can support implicit values.
This begins to turn cross-tests into a library with a defined interface. Existing cross-tests say that they are a prototype level API, and they feel that way. The prototyping was successful, and cross-tests have proven their worth. We should begin to move towards a more discover-able API. The first function of which is: ```go // crosstests/create.go func Create( t T, resource map[string]*schema.Schema, tfConfig cty.Value, puConfig resource.PropertyMap, options ...CreateOption, ) ``` We can continue to iterate on this API, but we should begin to formalize a boundary between the implementation of cross-tests and tests written with cross-tests. stack-info: PR: #2501, branch: iwahbe/stack/2
1c635d9
to
06c8af7
Compare
This begins to turn cross-tests into a library with a defined interface. Existing cross-tests say that they are a prototype level API, and they feel that way. The prototyping was successful, and cross-tests have proven their worth. We should begin to move towards a more discover-able API. The first function of which is: ```go // crosstests/create.go func Create( t T, resource map[string]*schema.Schema, tfConfig cty.Value, puConfig resource.PropertyMap, options ...CreateOption, ) ``` We can continue to iterate on this API, but we should begin to formalize a boundary between the implementation of cross-tests and tests written with cross-tests. stack-info: PR: #2501, branch: iwahbe/stack/2
06c8af7
to
bda8f93
Compare
This is reasonable, thanks:
Might need a way to set overrides (info.Schema aka SchemaInfo). As Venelin points out, some tests may want to omit specifying both tfConfig and puConfig if one can be unambiguously computed from the other. |
} | ||
|
||
// An option that can be used to customize [Create]. | ||
type CreateOption func(*createOpts) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For internal code I wonder sometimes if plain struct options are just easier to work with and discover than functional options. That's a stylistic nit though. It's all good.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, looks like it's getting a little nicer. Always welcome.
This begins to turn cross-tests into a library with a defined interface. Existing cross-tests say that they are a prototype level API, and they feel that way. The prototyping was successful, and cross-tests have proven their worth. We should begin to move towards a more discover-able API. The first function of which is: ```go // crosstests/create.go func Create( t T, resource map[string]*schema.Schema, tfConfig cty.Value, puConfig resource.PropertyMap, options ...CreateOption, ) ``` We can continue to iterate on this API, but we should begin to formalize a boundary between the implementation of cross-tests and tests written with cross-tests. stack-info: PR: #2501, branch: iwahbe/stack/2
bda8f93
to
563065f
Compare
563065f
to
4bd2817
Compare
Stacked PRs:
runCreateInputCheck
in terms ofCreate
#2503crosstests.Create
#2501Expose
crosstests.Create
This begins to turn cross-tests into a library with a defined interface. Existing
cross-tests say that they are a prototype level API, and they feel that way. The
prototyping was successful, and cross-tests have proven their worth. We should begin to
move towards a more discover-able API. The first function of which is:
We can continue to iterate on this API, but we should begin to formalize a boundary
between the implementation of cross-tests and tests written with cross-tests.