Test/binary canaries (#5710)

* chore: Update CFN Lint version to support python 3.11 (#5651)

* Update CFN Lint version to support python 3.11

* Run make update-reproducible-reqs

---------

Co-authored-by: Mohamed Elasmar <[email protected]>

* fix: Require file label to run before maintainers (#5656)

* Require file label to run before maintainers

* Run maintainer label job even if previous job fails

* Add Windows config

* Fix formatting

* Update config

* test

* Update config

* Update config

* Update config

* Update config

* chore: update aws_lambda_builders to 1.36.0 (#5655)

Co-authored-by: GitHub Action <[email protected]>
Co-authored-by: Mohamed Elasmar <[email protected]>

* Update config

* Update config

* fix(invoke): Write in UTF-8 string instead of bytes (#5642)

* Revert "fix: Revert UTF-8 fixes #5485 and #5427 (#5512)"

This reverts commit 36f8bf9.

* Enforce utf8 on stdout/stderr/logfile

---------

Co-authored-by: Jacob Fuss <[email protected]>

* chore(deps): bump cryptography from 41.0.2 to 41.0.3 in /requirements (#5675)

Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.2 to 41.0.3.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](pyca/cryptography@41.0.2...41.0.3)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add linux config

* fix(test): Force reading file with utf8 in tests for windows (#5679)

Co-authored-by: Jacob Fuss <[email protected]>

* fix(test): Increase max execution time for timeout tests (#5680)

Co-authored-by: Jacob Fuss <[email protected]>

* Update install links

* Update linux config

* Update linux config

* Update linux config

* Update linux config

* Update linux config

* Update linux config

* Update linux config

* Update linux config

* Update linux config

* Update linux config

* Update linux config

* chore: auto update reproducible requirements when there is a dependency change (#5677)

* chore: add reproducible gha

* format

* formatting v2

* format v3

* use win make target

* update windows job

* update win paths

* why windows why

* remove if stmt for now

* test with direct paths

* update windows paths again

* bring back activate

* add dummy win file and push changes

* update base.txt for testing

* Update reproducibles

* run jobs sequentially

* print changed flag

* check changes again

* run git status before moving forward

* refresh index before checking any changed files

* Update reproducibles

* commit or skip

* Update reproducibles

* rerun build & pyinstaller jobs once update reproducible finishes

* add more details to commit message

* nuke all reproducibles to trigger the CI

* Update reproducibles: update-reproducible-linux-reqs

* Update reproducibles: update-reproducible-mac-reqs

* Update reproducibles: update-reproducible-win-reqs

* change lb version for testing

* update permissions and limit only for aws/aws-sam-cli

* Update reproducibles: update-reproducible-linux-reqs

* Update reproducibles: update-reproducible-mac-reqs

* Update reproducibles: update-reproducible-win-reqs

* update other jobs as well

* run without file filter

* Update reproducibles: update-reproducible-linux-reqs

* Update reproducibles: update-reproducible-mac-reqs

* Update reproducibles: update-reproducible-win-reqs

* put the file filter back

---------

Co-authored-by: GitHub Action <[email protected]>

* fix: Repair failing integration test (#5698)

* Fix failing integration test

* Be more specific with error message

---------

Co-authored-by: Leonardo Gama <[email protected]>

* Update Windows config

* fix: link the API gateway resource parent to either rest api or another gateway resource (#5697)

* Update Windows config

* Update Windows config

* Update Windows config

* Update Windows config

* Update Windows config

* Update Windows config

* Update Windows config

* Update Windows config

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Connor Kirkpatrick <[email protected]>
Co-authored-by: Mohamed Elasmar <[email protected]>
Co-authored-by: Lucas <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Action <[email protected]>
Co-authored-by: Jacob Fuss <[email protected]>
Co-authored-by: Jacob Fuss <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mehmet Nuri Deveci <[email protected]>
Co-authored-by: Leonardo Gama <[email protected]>
Co-authored-by: Leonardo Gama <[email protected]>

tests/integration/telemetry/test_experimental_metric.py

@@ -7,6 +7,7 @@
 
 from .integ_base import IntegBase, TelemetryServer
 from samcli import __version__ as SAM_CLI_VERSION
+from ...testing_utils import strip_nightly_installer_suffix
 
 
 class TestExperimentalMetric(IntegBase):
@@ -162,6 +163,7 @@ def test_must_send_cdk_project_type_metrics(self):
             for req in all_requests:
                 if "commandRun" in req["data"]["metrics"][0]:
                     request = req  # We're only testing the commandRun metric
+            strip_nightly_installer_suffix(request, "commandRun")
             self.assertIn("Content-Type", request["headers"])
             self.assertEqual(request["headers"]["Content-Type"], "application/json")
 
@@ -216,6 +218,7 @@ def test_must_send_not_experimental_metrics_if_not_experimental(self):
             # commandRun metric will be first or second, so we sort for consistency.
             all_requests.sort(key=lambda x: list(x["data"]["metrics"][0].keys())[0])
             request = all_requests[0]  # "commandRun" comes before "events"
+            strip_nightly_installer_suffix(request, "commandRun")
             self.assertIn("Content-Type", request["headers"])
             self.assertEqual(request["headers"]["Content-Type"], "application/json")
 

tests/integration/telemetry/test_installed_metric.py

@@ -3,6 +3,7 @@
 from unittest.mock import ANY
 from .integ_base import IntegBase, TelemetryServer, EXPECTED_TELEMETRY_PROMPT
 from samcli import __version__ as SAM_CLI_VERSION
+from ...testing_utils import strip_nightly_installer_suffix
 
 
 class TestSendInstalledMetric(IntegBase):
@@ -32,6 +33,7 @@ def test_send_installed_metric_on_first_run(self):
             requests = filter_installed_metric_requests(all_requests)
             self.assertEqual(1, len(requests), "There should be only one 'installed' metric")
             request = requests[0]
+            strip_nightly_installer_suffix(request, "installed")
             self.assertIn("Content-Type", request["headers"])
             self.assertEqual(request["headers"]["Content-Type"], "application/json")
 
@@ -113,11 +115,6 @@ def filter_installed_metric_requests(all_requests):
     for r in all_requests:
         data = r["data"]
         if "metrics" in data and data["metrics"] and "installed" in data["metrics"][0]:
-            # If it's a nightly release version, it will have a suffix.
-            # We can strip it for the purpose of testing telemetry
-            version = data["metrics"][0].get("installed", {}).get("samcliVersion", "")
-            if version:
-                data["metrics"][0]["installed"]["samcliVersion"] = version[: len(SAM_CLI_VERSION)]
             result.append(r)
 
     return result

tests/testing_utils.py

@@ -11,6 +11,8 @@
 from subprocess import Popen, PIPE, TimeoutExpired
 from queue import Queue
 
+from samcli import __version__ as SAM_CLI_VERSION
+
 import shutil
 from uuid import uuid4
 
@@ -243,6 +245,19 @@ def _get_current_account_id():
     return account_id
 
 
+def strip_nightly_installer_suffix(request: dict, metric_type: str):
+    """
+    If it's a nightly release version, it will have a suffix.
+    We can strip it for the purpose of testing telemetry.
+    """
+    metrics = request.get("data", {}).get("metrics", [])
+    if not metrics:
+        return
+    version = metrics[0].get(metric_type, {}).get("samcliVersion", "")
+    if version:
+        request["data"]["metrics"][0][metric_type]["samcliVersion"] = version[: len(SAM_CLI_VERSION)]
+
+
 class UpdatableSARTemplate:
     """
     This class is used to replace the `${AWS::AccountId}` in the testing templates with the account id for the testing