Fix merge conflict in release 1.29.0 (#1771)

* Fix: Updated Slack Invite Link (#1712) * Updated Slack Invite Link * Restricted jsonschema to Python 2 * Forced pyrsistent to 0.16 in Python 2 * Reverted Changes to enum34 * Merge master back to develop (#1734) * Release v1.26.0 (#1680) * feat: add support for VPCEndpointIds in EndpointConfiguration * fix: update formatting with black * docs: update 2016-10-31.md * docs: added api endpointconfiguration example * docs: make example more generic * fix: remove nested EndpointConfiguration types from output * fix: only allow one EndpointConfiguration Type * doc: update example to reflect only allowing one EndpointConfiguration Type * fix : missing UserPool properties (#1506) (#1581) * fix: resource policy generation for {path+} (#1580) * refactor: Remove 2016-10-31 examples * update PR template * adjust pr template * Adding authorization scopes as list validation in ApiGatewayAuthorizer (v1 and v2). (#1670) * Adding authorization scopes as list validation in ApiGatewayAuthorizer and ApiGatewayV2Authorizer. * make black. * Adding functional test for invalid auth scope. * adding error condition for invalid test. * removing test template file. * feat: MSK event type support for AWS::Serverless::Function (#52) Co-authored-by: Steve Brown <[email protected]> Co-authored-by: jtaylor00 <[email protected]> Co-authored-by: Jacob Fuss <[email protected]> Co-authored-by: Alex Wood <[email protected]> Co-authored-by: Tarun <[email protected]> * Update __init__.py (#1704) * Release/v1.27.0 resolveconflict (#1717) * Release v1.26.0 (#1680) * feat: add support for VPCEndpointIds in EndpointConfiguration * fix: update formatting with black * docs: update 2016-10-31.md * docs: added api endpointconfiguration example * docs: make example more generic * fix: remove nested EndpointConfiguration types from output * fix: only allow one EndpointConfiguration Type * doc: update example to reflect only allowing one EndpointConfiguration Type * fix : missing UserPool properties (#1506) (#1581) * fix: resource policy generation for {path+} (#1580) * refactor: Remove 2016-10-31 examples * update PR template * adjust pr template * Adding authorization scopes as list validation in ApiGatewayAuthorizer (v1 and v2). (#1670) * Adding authorization scopes as list validation in ApiGatewayAuthorizer and ApiGatewayV2Authorizer. * make black. * Adding functional test for invalid auth scope. * adding error condition for invalid test. * removing test template file. * feat: MSK event type support for AWS::Serverless::Function (#52) Co-authored-by: Steve Brown <[email protected]> Co-authored-by: jtaylor00 <[email protected]> Co-authored-by: Jacob Fuss <[email protected]> Co-authored-by: Alex Wood <[email protected]> Co-authored-by: Tarun <[email protected]> * Fix: Updated Slack Invite Link (#1712) * Updated Slack Invite Link * Restricted jsonschema to Python 2 * Forced pyrsistent to 0.16 in Python 2 * Reverted Changes to enum34 Co-authored-by: Sriram Madapusi Vasudevan <[email protected]> Co-authored-by: Steve Brown <[email protected]> Co-authored-by: jtaylor00 <[email protected]> Co-authored-by: Jacob Fuss <[email protected]> Co-authored-by: Alex Wood <[email protected]> Co-authored-by: Tarun <[email protected]> Co-authored-by: Cosh_ <[email protected]> Co-authored-by: Sriram Madapusi Vasudevan <[email protected]> Co-authored-by: Steve Brown <[email protected]> Co-authored-by: jtaylor00 <[email protected]> Co-authored-by: Jacob Fuss <[email protected]> Co-authored-by: Alex Wood <[email protected]> Co-authored-by: Tarun <[email protected]> Co-authored-by: Mehmet Nuri Deveci <[email protected]> Co-authored-by: Cosh_ <[email protected]> * Lambdaauth (#1733) * Add support for Lambda Authorizers in HttpAPI * Address comments and fix formatting * fix version * Validate input parameters. Update tests * black reformat Co-authored-by: Raymond Wang <[email protected]> * Feature toggle (#1737) * Adding logic to pipe app config providers. Unit test pending * Adding some documentation to config providers. * Adding some unit tests and making black ignore json files. * minor cleanup. * Addressing PR comments. * feature: Support MTLS auth properties in REST and HTTP API domain names (#1725) * feature: Support MTLS auth properties in REST and HTTP API domain names * fix unicode != str issue in py2.7 * add SecurityPolicy because default RESTAPI is using TLS1.0 * Add new property DisableExecuteApiEndpoint * black reformat * Address comments * Add tests on invalid templates * address test failures in py2.7 * restart travis tests * fix: adding support for passing target id to EventBridgeRule (#1747) * Manage black version using requirement file (#1748) * chore: Manage black version in dev.txt - config pre-commit - config development guide - config travis Refer to the commit below in aws-sam-cli aws/aws-sam-cli@d725db5fbfc698a9f0c7582 * Format using black 20.8b1 * Opt-out black fron dev.txt for Python 2 * Release/v1.29.0 (#1769) * Mq event source (#60) * Support AmazonMQ as event source * Set black hook language version to python3 * chore: version bump (#64) * Black reformat Co-authored-by: Kaidi He <[email protected]> Co-authored-by: Cosh_ <[email protected]> Co-authored-by: Raymond Wang <[email protected]> Co-authored-by: Sriram Madapusi Vasudevan <[email protected]> Co-authored-by: Steve Brown <[email protected]> Co-authored-by: jtaylor00 <[email protected]> Co-authored-by: Jacob Fuss <[email protected]> Co-authored-by: Alex Wood <[email protected]> Co-authored-by: Tarun <[email protected]> Co-authored-by: Mehmet Nuri Deveci <[email protected]> Co-authored-by: Tolledo <[email protected]> Co-authored-by: _sam <[email protected]> Co-authored-by: Kaidi He <[email protected]>
aws · Nov 5, 2020 · 815b6aa · 815b6aa
1 parent df07ee4
commit 815b6aa
Show file tree

Hide file tree

Showing 69 changed files with 1,120 additions and 125 deletions.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -1,7 +1,9 @@
+# https://pre-commit.com/#repository-local-hooks
 repos:
-  - repo: https://github.com/python/black
-    rev: 19.3b0
-    hooks:
-      - id: black
-        language_version: python3.7
-        exclude_types: ['markdown', 'ini', 'toml', 'rst']
+- repo: local
+  hooks:
+    - id: black
+      name: black
+      entry: black
+      language: system
+      types: [python]
diff --git a/.travis.yml b/.travis.yml
@@ -21,12 +21,6 @@ matrix:
 
 install:
 # Install the code requirements
-- mkdir $HOME/bin-black
-- wget -O $HOME/bin-black/black https://github.com/python/black/releases/download/19.10b0/black
-- chmod +x $HOME/bin-black/black
-- export PATH=$PATH:$HOME/bin-black
-- black --version
-
 - make init
 
 # Install Docs requirements

diff --git a/DEVELOPMENT_GUIDE.rst b/DEVELOPMENT_GUIDE.rst
@@ -28,16 +28,44 @@ Setup Python locally using `pyenv`_
 
 2. Install Additional Tooling
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-1. Black
-~~~~~~~~
+Black
+'''''
 We format our code using `Black`_ and verify the source code is black compliant
-in Appveyor during PRs. You can find installation instructions on `Black's docs`_.
+in Appveyor during PRs. Black will be installed automatically with ``make init``.
 
-After installing, you can check your formatting through our Makefile by running `make black-check`. To automatically update your code to match our formatting, please run `make black`. You can also integrate Black directly in your favorite IDE (instructions
-can be found `here`_)
+After installing, you can run our formatting through our Makefile by
+``make black`` or integrating Black directly in your favorite IDE
+(instructions can be found `here <https://black.readthedocs.io/en/stable/editor_integration.html>`__)
+
+(workaround) Integrating Black directly in your favorite IDE
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+Since black is installed in virtualenv, when you follow `this
+instruction <https://black.readthedocs.io/en/stable/editor_integration.html>`__,
+``which black`` might give you this
+
+::
+
+   (samtranslator37) $ where black
+   /Users/<username>/.pyenv/shims/black
+
+However, IDEs such as PyCharm (using FileWatcher) will have a hard time
+invoking ``/Users/<username>/.pyenv/shims/black`` and this will happen:
+
+::
+
+   pyenv: black: command not found
+
+   The `black' command exists in these Python versions:
+     3.7.2/envs/samtranslator37
+     samtranslator37
+
+A simple workaround is to use
+``/Users/<username>/.pyenv/versions/samtranslator37/bin/black`` instead of
+``/Users/<username>/.pyenv/shims/black``.
 
 Pre-commit
-~~~~~~~~~~
+''''''''''
 If you don't wish to manually run black on each pr or install black manually, we have integrated black into git hooks through `pre-commit`_.
 After installing pre-commit, run `pre-commit install` in the root of the project. This will install black for you and run the black formatting on
 commit.

diff --git a/Makefile b/Makefile
@@ -9,17 +9,20 @@ test:
 	pytest --cov samtranslator --cov-report term-missing --cov-fail-under 95 tests
 
 black:
-	black setup.py samtranslator/* tests/* bin/*
+	black setup.py samtranslator/* tests/* bin/*.py
 
 black-check:
-	black --check setup.py samtranslator/* tests/* bin/*
+	black --check setup.py samtranslator/* tests/* bin/*.py
 
 # Command to run everytime you make changes to verify everything works
 dev: test
 
 # Verifications to run before sending a pull request
 pr: black-check init dev
 
+# Verifications to run before sending a pull request, skipping black check because black requires Python 3.6+
+pr2.7: init dev
+
 define HELP_MESSAGE
 
 Usage: $ make [TARGETS]

diff --git a/bin/sam-translate.py b/bin/sam-translate.py
@@ -35,6 +35,7 @@
 from samtranslator.translator.transform import transform
 from samtranslator.yaml_helper import yaml_parse
 from samtranslator.model.exceptions import InvalidDocumentException
+from samtranslator.feature_toggle.feature_toggle import FeatureToggleLocalConfigProvider, FeatureToggle
 
 LOG = logging.getLogger(__name__)
 cli_options = docopt(__doc__)
@@ -95,7 +96,12 @@ def transform_template(input_file_path, output_file_path):
         sam_template = yaml_parse(f)
 
     try:
-        cloud_formation_template = transform(sam_template, {}, ManagedPolicyLoader(iam_client))
+        feature_toggle = FeatureToggle(
+            FeatureToggleLocalConfigProvider(
+                os.path.join(my_path, "..", "tests", "feature_toggle", "input", "feature_toggle_config.json")
+            )
+        )
+        cloud_formation_template = transform(sam_template, {}, ManagedPolicyLoader(iam_client), feature_toggle)
         cloud_formation_template_prettified = json.dumps(cloud_formation_template, indent=2)
 
         with open(output_file_path, "w") as f:

diff --git a/docs/cloudformation_compatibility.rst b/docs/cloudformation_compatibility.rst
@@ -108,6 +108,16 @@ StartingPosition         All
 BatchSize                All
 ======================== ================================== ========================
 
+MQ
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+======================== ================================== ========================
+     Property Name        Intrinsic(s) Supported            Reasons
+======================== ================================== ========================
+Broker                          All
+Queues                          All
+SourceAccessConfigurations      All
+======================== ================================== ========================
+
 MSK
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 ======================== ================================== ========================

diff --git a/docs/internals/generated_resources.rst b/docs/internals/generated_resources.rst
@@ -326,6 +326,36 @@ AWS::Lambda::Permission            MyFunction\ **MyTrigger**\ Permission
 AWS::Lambda::EventSourceMapping    MyFunction\ **MyTrigger** 
 ================================== ================================
 
+MQ
+^^^^^^^
+
+Example:
+
+.. code:: yaml
+
+  MyFunction:
+    Type: AWS::Serverless::Function
+    Properties:
+      ...
+      Events:
+        MyTrigger:
+          Type: MQ
+          Properties:
+            Broker: arn:aws:mq:us-east-2:123456789012:broker:MyBroker:b-1234a5b6-78cd-901e-2fgh-3i45j6k178l9
+            SourceAccessConfigurations:
+              Type: BASIC_AUTH
+              URI: arn:aws:secretsmanager:us-west-2:123456789012:secret:my-path/my-secret-name-1a2b3c
+      ...
+
+Additional generated resources:
+
+================================== ================================
+CloudFormation Resource Type       Logical ID
+================================== ================================
+AWS::Lambda::Permission            MyFunction\ **MyTrigger**\ Permission
+AWS::Lambda::EventSourceMapping    MyFunction\ **MyTrigger**
+================================== ================================
+
 MSK
 ^^^^^^^
 

diff --git a/requirements/dev.txt b/requirements/dev.txt
@@ -15,3 +15,6 @@ requests>=2.20.0
 
 # CLI requirements
 docopt>=0.6.2
+
+# formatter
+black==20.8b1; python_version >= '3.6'
diff --git a/samtranslator/__init__.py b/samtranslator/__init__.py
@@ -1 +1 @@
-__version__ = "1.28.1"
+__version__ = "1.29.0"
diff --git a/samtranslator/feature_toggle/__init__.py b/samtranslator/feature_toggle/__init__.py
diff --git a/samtranslator/feature_toggle/feature_toggle.py b/samtranslator/feature_toggle/feature_toggle.py
@@ -0,0 +1,125 @@
+import os
+import sys
+import json
+import boto3
+import logging
+
+my_path = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, my_path + "/..")
+
+LOG = logging.getLogger(__name__)
+
+
+class FeatureToggle:
+    """
+    FeatureToggle is the class which will provide methods to query and decide if a feature is enabled based on where
+    SAM is executing or not.
+    """
+
+    def __init__(self, config_provider):
+        self.feature_config = config_provider.config
+
+    def is_enabled_for_stage_in_region(self, feature_name, stage, region="default"):
+        """
+        To check if feature is available for a particular stage or not.
+        :param feature_name: name of feature
+        :param stage: stage where SAM is running
+        :param region: region in which SAM is running
+        :return:
+        """
+        if feature_name not in self.feature_config:
+            LOG.warning("Feature '{}' not available in Feature Toggle Config.".format(feature_name))
+            return False
+        stage_config = self.feature_config.get(feature_name, {}).get(stage, {})
+        if not stage_config:
+            LOG.info("Stage '{}' not enabled for Feature '{}'.".format(stage, feature_name))
+            return False
+        region_config = stage_config.get(region, {}) if region in stage_config else stage_config.get("default", {})
+        is_enabled = region_config.get("enabled", False)
+        LOG.info("Feature '{}' is enabled: '{}'".format(feature_name, is_enabled))
+        return is_enabled
+
+    def is_enabled_for_account_in_region(self, feature_name, stage, account_id, region="default"):
+        """
+        To check if feature is available for a particular account or not.
+        :param feature_name: name of feature
+        :param stage: stage where SAM is running
+        :param account_id: account_id who is executing SAM template
+        :param region: region in which SAM is running
+        :return:
+        """
+        if feature_name not in self.feature_config:
+            LOG.warning("Feature '{}' not available in Feature Toggle Config.".format(feature_name))
+            return False
+        stage_config = self.feature_config.get(feature_name, {}).get(stage, {})
+        if not stage_config:
+            LOG.info("Stage '{}' not enabled for Feature '{}'.".format(stage, feature_name))
+            return False
+        account_config = stage_config.get(account_id) if account_id in stage_config else stage_config.get("default", {})
+        region_config = (
+            account_config.get(region, {}) if region in account_config else account_config.get("default", {})
+        )
+        is_enabled = region_config.get("enabled", False)
+        LOG.info("Feature '{}' is enabled: '{}'".format(feature_name, is_enabled))
+        return is_enabled
+
+
+class FeatureToggleConfigProvider:
+    """Interface for all FeatureToggle config providers"""
+
+    def __init__(self):
+        pass
+
+    @property
+    def config(self):
+        raise NotImplementedError
+
+
+class FeatureToggleDefaultConfigProvider(FeatureToggleConfigProvider):
+    """Default config provider, always return False for every query."""
+
+    def __init__(self):
+        FeatureToggleConfigProvider.__init__(self)
+
+    @property
+    def config(self):
+        return {}
+
+
+class FeatureToggleLocalConfigProvider(FeatureToggleConfigProvider):
+    """Feature toggle config provider which uses a local file. This is to facilitate local testing."""
+
+    def __init__(self, local_config_path):
+        FeatureToggleConfigProvider.__init__(self)
+        with open(local_config_path, "r") as f:
+            config_json = f.read()
+        self.feature_toggle_config = json.loads(config_json)
+
+    @property
+    def config(self):
+        return self.feature_toggle_config
+
+
+class FeatureToggleAppConfigConfigProvider(FeatureToggleConfigProvider):
+    """Feature toggle config provider which loads config from AppConfig."""
+
+    def __init__(self, application_id, environment_id, configuration_profile_id):
+        FeatureToggleConfigProvider.__init__(self)
+        self.app_config_client = boto3.client("appconfig")
+        try:
+            response = self.app_config_client.get_configuration(
+                Application=application_id,
+                Environment=environment_id,
+                Configuration=configuration_profile_id,
+                ClientId="FeatureToggleAppConfigConfigProvider",
+            )
+            binary_config_string = response["Content"].read()
+            self.feature_toggle_config = json.loads(binary_config_string.decode("utf-8"))
+        except Exception as ex:
+            LOG.error("Failed to load config from AppConfig: {}. Using empty config.".format(ex))
+            # There is chance that AppConfig is not available in a particular region.
+            self.feature_toggle_config = json.loads("{}")
+
+    @property
+    def config(self):
+        return self.feature_toggle_config
diff --git a/samtranslator/model/__init__.py b/samtranslator/model/__init__.py
@@ -355,8 +355,7 @@ def to_cloudformation(self, **kwargs):
 
 
 class SamResourceMacro(ResourceMacro):
-    """ResourceMacro that specifically refers to SAM (AWS::Serverless::*) resources.
-    """
+    """ResourceMacro that specifically refers to SAM (AWS::Serverless::*) resources."""
 
     # SAM resources can provide a list of properties that they expose. These properties usually resolve to
     # CFN resources that this SAM resource generates. This is provided as a map with the following format:

diff --git a/samtranslator/model/api/api_generator.py b/samtranslator/model/api/api_generator.py
@@ -304,6 +304,37 @@ def _construct_api_domain(self, rest_api):
 
         domain.EndpointConfiguration = {"Types": [endpoint]}
 
+        mutual_tls_auth = self.domain.get("MutualTlsAuthentication", None)
+        if mutual_tls_auth:
+            if isinstance(mutual_tls_auth, dict):
+                if not set(mutual_tls_auth.keys()).issubset({"TruststoreUri", "TruststoreVersion"}):
+                    invalid_keys = list()
+                    for key in mutual_tls_auth.keys():
+                        if not key in {"TruststoreUri", "TruststoreVersion"}:
+                            invalid_keys.append(key)
+                    invalid_keys.sort()
+                    raise InvalidResourceException(
+                        ",".join(invalid_keys),
+                        "Available MutualTlsAuthentication fields are {}.".format(
+                            ["TruststoreUri", "TruststoreVersion"]
+                        ),
+                    )
+                domain.MutualTlsAuthentication = {}
+                if mutual_tls_auth.get("TruststoreUri", None):
+                    domain.MutualTlsAuthentication["TruststoreUri"] = mutual_tls_auth["TruststoreUri"]
+                if mutual_tls_auth.get("TruststoreVersion", None):
+                    domain.MutualTlsAuthentication["TruststoreVersion"] = mutual_tls_auth["TruststoreVersion"]
+            else:
+                raise InvalidResourceException(
+                    mutual_tls_auth,
+                    "MutualTlsAuthentication must be a map with at least one of the following fields {}.".format(
+                        ["TruststoreUri", "TruststoreVersion"]
+                    ),
+                )
+
+        if self.domain.get("SecurityPolicy", None):
+            domain.SecurityPolicy = self.domain["SecurityPolicy"]
+
         # Create BasepathMappings
         if self.domain.get("BasePath") and isinstance(self.domain.get("BasePath"), string_types):
             basepaths = [self.domain.get("BasePath")]