Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FlatteningError while working with remote "$ref" in typeConfiguration #1084

Open
Scribbd opened this issue Jul 29, 2024 · 2 comments
Open

FlatteningError while working with remote "$ref" in typeConfiguration #1084

Scribbd opened this issue Jul 29, 2024 · 2 comments

Comments

@Scribbd
Copy link

Scribbd commented Jul 29, 2024

I am working on a set of custom resource types that share two pieces of configuration over all planned types:

  • The Environment property
  • The EnvironmentMapping typeConfiguration

To limit the amount of duplicate JSON definitions, I took a page out of the teams book and created a bucket where I placed my remote definitions. And uploaded the shared definitions in that bucket.

Resulting in a definition JSON like follows:

{
    "$schema": "https://schema.cloudformation.us-east-1.amazonaws.com/provider.definition.schema.v1.json#",
    "properties": {
        "Title": {
            "description": "Title/Name of the folder.",
            "type": "string"
        },
        "Environment": {
            "$ref": "https://s3.eu-west-1.amazonaws.com/my.fake.schemas.bucket/environment.def.v0.json"
        }
    },
...
   "typeConfiguration": {
        "additionalProperties": false,
        "properties": {
            "EnvironmentMapping":{
                "description": "A list of environment definitions detailing which secret is bound to which host.",
                "type":"array",
                "insertionOrder": false,
                "items": {
                    "$ref": "https://s3.eu-west-1.amazonaws.com/my.fake.schemas.bucket/environment-config.def.v0.json"
                }
            }
        },
        "required": ["EnvironmentConfiguration"]
    }
}

It works for the Environment property. That by itself and cfn generate finishes without problem. But when I add the reference to the typeConfiguration, cfn generate will produce the following error:
rpdk.core.jsonutils.utils.FlatteningError: Invalid ref: ('remote', 'schema1')

When perusing through the logs, I do see both schema's being picked up and verified. Both written to remote/schema0 and remote/schema1. And files are generated, but the config JSON looks as follows:

{
    "additionalProperties": false,
    "properties": {
        "EnvironmentConfiguration": {
            "type": "array",
            "insertionOrder": false,
            "items": {
                "$ref": "#/remote/schema1"
            }
        }
    },
    "required": [
        "EnvironmentConfiguration"
    ],
    "definitions": {},
    "typeName": "ORG::App::Folder"
}

I tried a few things:

  • I swapped both $ref-s: Flattening error. Resulting generated code did reflect the swapped configuration.
  • I added an extra object layer to the config definition json: Flattening error.
  • Move external property into the definitions block: Works, generation successful. But it defeats the effort of keeping the amount of copied schema low.

I might dive into the code and see how both blocks are rendered and how they differ. But it is late for me, at the time of writing this issue, and maybe someone will see an obvious mistake I missed.

Thanks in advance.

@Scribbd
Copy link
Author

Scribbd commented Jul 31, 2024

There is a pragmatic fix:

diff --git a/src/rpdk/core/project.py b/src/rpdk/core/project.py
index bbf0922..b0870ae 100644
--- a/src/rpdk/core/project.py
+++ b/src/rpdk/core/project.py
@@ -509,6 +509,7 @@ class Project:  # pylint: disable=too-many-instance-attributes,too-many-public-m
         if "typeConfiguration" in self.schema:
             configuration_schema = self.schema["typeConfiguration"]
             configuration_schema["definitions"] = self.schema.get("definitions", {})
+            configuration_schema["remote"] = self.schema.get("remote", {})
             configuration_schema["typeName"] = self.type_name
             self.configuration_schema = configuration_schema

By simply adding the remote definitions to the to-be generated type-configuration. It will pass the flattening check, and still be valid according to the meta-schema which allows for a remote key.

I was unable to find a meta-schema specific for the type-configuration. So, I am assuming that the provider one applies here as well.

@Scribbd
Copy link
Author

Scribbd commented Aug 19, 2024

I found the appropriate schema for type configuration: https://github.com/aws-cloudformation/cloudformation-cli/blob/master/src/rpdk/core/data/schema/provider.configuration.definition.schema.v1.json
This one does not allow for the 'remote'-attribute. I will be closing the pull request as my 'pragmatic' fix won't be compliant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant