Add config file support for control omission (#431)

* Update Config.md with section on omitting policies * Create omit_policies.yaml * Update Config.md with example config specific to scubagoggles * Update Config.md minor grammatical changes * Update Config.md correct link * Update Config.md correct minor typo * Add config file support for omitting policies * Correct some linter errors * Correct more linter errors * Move pylint disable comment * Update Config.md, clarify usage of single config file * Update omit_policies.yaml, comment about usage of anchors * Warn for unexpected control ids * must appease the linter * Remove commented out code Co-authored-by: David Bui <[email protected]> * Break out omitpolicy validation into separate function. * Add final newline to file for the linter * Add special handling for rules * Remove duplicate word Co-authored-by: mitchelbaker-cisa <[email protected]> * Create warn wrapper function to clean tqdm output --------- Co-authored-by: David Bui <[email protected]> Co-authored-by: mitchelbaker-cisa <[email protected]>
cisagov · Oct 9, 2024 · 8cc7ff3 · 8cc7ff3
1 parent c4258ae
commit 8cc7ff3
Show file tree

Hide file tree

Showing 7 changed files with 316 additions and 73 deletions.
diff --git a/docs/usage/Config.md b/docs/usage/Config.md
@@ -6,12 +6,12 @@ All ScubaGoggles [parameters](/docs/usage/Parameters.md) can be placed into a co
 > If a parameter is specified both on the command-line and in a configuration file, the command-line parameter has precedence over the config file.
 
 ## Sample Configuration Files
-[Sample config files](/sample-config-files) are available in the repo and are discussed below.
+[Sample config files](/sample-config-files) are available in the repo and are discussed below. When executing ScubaGoggles, only a single config file can be read in; we recommend looking through the following examples and constructing a config file that best suits your use case.
 
 ### Basic Usage
 The [basic use](/sample-config-files/basic_config.yaml) example config file specifies the `outpath`, `baselines`, and `quiet` parameters.
 
-ScubaGoggles can be invokes with this config file:
+ScubaGoggles can be invoked with this config file:
 ```
 scubagoggles gws --config basic_config.yaml
 ```
@@ -21,6 +21,19 @@ It can also be invoked while overriding the `baselines` parameter.
 scubagoggles gws --config basic_config.yaml -b gmail chat
 ```
 
+### Omit Policies
+
+In some cases, it may be appropriate to omit specific policies from ScubaGoggles evaluation. For example:
+- When a policy is implemented by a third-party service that ScubaGoggles does not audit.
+- When a policy is not applicable to your organization (e.g., policy GWS.GMAIL.4.3v0.3, which is only applicable to federal, executive branch, departments and agencies).
+
+The `omitpolicy` top-level key, shown in this [example ScubaGoggles configuration file](/sample-config-files/omit_policies.yaml), allows the user to specify the policies that should be omitted from the ScubaGoggles report. Omitted policies will show up as "Omitted" in the HTML report and will be colored gray. Omitting policies must only be done if the omissions are approved within an organization's security risk management process. **Exercise care when omitting policies because this can inadvertently introduce blind spots when assessing your system.**
+
+For each omitted policy, the config file allows you to indicate the following:
+- `rationale`: The reason the policy should be omitted from the report. This value will be displayed in the "Details" column of the report. ScubaGoggles will output a warning if no rationale is provided.
+- `expiration`: Optional. A date after which the policy should no longer be omitted from the report. The expected format is yyyy-mm-dd.
+
+
 ## Navigation
 - Continue to [Usage: Examples](/docs/usage/Examples.md)
-- Return to [Documentation Home](/README.md)
+- Return to [Documentation Home](/README.md)
diff --git a/sample-config-files/omit_policies.yaml b/sample-config-files/omit_policies.yaml
@@ -0,0 +1,22 @@
+# YAML configuration demonstrating omitting policies from ScubaGoggles evaluation.
+# Any omitted policies should be carefully considered and documented as part of an
+# organization's cybersecurity risk management program process and practices.
+
+baselines: [gmail, chat, drive]
+
+omitpolicy:
+  GWS.GMAIL.3.1v0.3:
+    rationale: "Known false positive; our SPF policy currently cannot to be retrieved via ScubaGoggles due to a split
+      horizon setup but is available publicly."
+    expiration: "2023-12-31"
+  GWS.CHAT.5.1v0.3:
+    rationale: &DLPRationale "The DLP capability required by the baselines is implemented by third party product, [x],
+      which ScubaGoggles does not have the ability to check."
+  GWS.DRIVEDOCS.7.1v0.3:
+    rationale: *DLPRationale
+
+# The "&" character used in the above example defines an anchor, which saves a value
+# for future reference. This value can then be retrieved with the "*" character. See
+# https://yaml.org/spec/1.2.2/#692-node-anchors for more details. In this case, the
+# anchor allows you to configure multiple omissions that share the same rationale
+# without repeating yourself.
diff --git a/scubagoggles/orchestrator.py b/scubagoggles/orchestrator.py
@@ -155,9 +155,10 @@ def _generate_summary(cls, stats: dict) -> str:
         n_fail = stats["Failures"]
         n_manual = stats["Manual"]
         n_error = stats["Errors"]
+        n_omit = stats['Omit']
 
         pass_summary = (f"<div class='summary pass'>{n_success}"
-        f" {cls._pluralize('test', 'tests', n_success)} passed</div>")
+        f" {cls._pluralize('pass', 'passes', n_success)}</div>")
 
         # The warnings, failures, and manuals are only shown if they are
         # greater than zero. Reserve the space for them here. They will
@@ -166,21 +167,26 @@ def _generate_summary(cls, stats: dict) -> str:
         failure_summary = "<div class='summary'></div>"
         manual_summary = "<div class='summary'></div>"
         error_summary = "<div class='summary'></div>"
+        omit_summary = "<div class='summary'></div>"
 
         if n_warn > 0:
             warning_summary = (f"<div class='summary warning'>{n_warn}"
             f" {cls._pluralize('warning', 'warnings', n_warn)}</div>")
         if n_fail > 0:
             failure_summary = (f"<div class='summary failure'>{n_fail}"
-            f" {cls._pluralize('test', 'tests', n_fail)} failed</div>")
+            f" {cls._pluralize('failure', 'failures', n_fail)}</div>")
         if n_manual > 0:
             manual_summary = (f"<div class='summary manual'>{n_manual} manual"
-            f" {cls._pluralize('check', 'checks', n_manual)} needed</div>")
+            f" {cls._pluralize('check', 'checks', n_manual)}</div>")
         if n_error > 0:
             error_summary = (f"<div class='summary error'>{n_error}"
             f" {cls._pluralize('error', 'errors', n_error)}</div>")
+        if n_omit > 0:
+            omit_summary = (f"<div class='summary manual'>{n_omit}"
+            " omitted</div>")
 
-        return f"{pass_summary}{warning_summary}{failure_summary}{manual_summary}{error_summary}"
+        return f"{pass_summary}{warning_summary}{failure_summary}" \
+            f"{manual_summary}{omit_summary}{error_summary}"
 
     def _run_reporter(self):
         """
@@ -245,6 +251,11 @@ def _run_reporter(self):
             tenant_info = json.load(file)['tenant_info']
             tenant_domain = tenant_info['domain']
 
+        # Determine if any controls were omitted in the config file
+        omissions = {}
+        if 'omitpolicy' in args and args.omitpolicy is not None:
+            omissions = args.omitpolicy
+
         # Create the individual report files
         out_jsonfile = args.outjsonfilename
         summary = {}
@@ -284,7 +295,9 @@ def _run_reporter(self):
                                 prod_to_fullname,
                                 baseline_policies[product],
                                 successful_calls,
-                                unsuccessful_calls)
+                                unsuccessful_calls,
+                                omissions,
+                                products_bar)
             stats_and_data[product] = \
                 reporter.rego_json_to_ind_reports(test_results_data,
                                                   out_folder)