Merge pull request #268 from splunk/CRL-1645

CRL-1645 - New detection "Abnormally High AWS Instances Terminated by…

baselines/train_ec2_excessive_terminateinstances.yml

@@ -0,0 +1,60 @@
+baseline:
+  splunk:
+    macros:
+      - ec2_excessive_terminateinstances_mltk_input_filter
+    schedule:
+      cron_schedule: ''
+      earliest_time: -91d@d
+      latest_time: -1d@d
+    search: >-
+      sourcetype=aws:cloudtrail eventName=TerminateInstances errorCode=success `ec2_excessive_terminateinstances_mltk_input_filter`
+      | bucket span=10m _time
+      | stats count as instances_terminated by _time src_user
+      | fit DensityFunction instances_terminated threshold=0.0005 into ec2_excessive_terminateinstances_v1
+creation_date: '2019-11-14'
+data_metadata:
+  data_source:
+    - AWS CloudTrail logs
+  data_sourcetypes:
+    - aws:cloudtrail
+  providing_technologies:
+    - AWS
+description: This search is used to build a Machine Learning Toolkit (MLTK) model
+  for how many TerminateInstances users do in the environment. By default, the search 
+  uses the last 90 days of data to build the model. The model created by this search 
+  is then used in the corresponding detection search, which identifies subsequent 
+  outliers in the number of TerminateInstances performed by a user in a small time window.
+eli5: Create a machine-learning (ML) model to establish a baseline for how many 
+  TerminateInstances users do in the environment. This can help you identify excessive
+  numbers of TerminateInstances which may warrant further investigation to determine if there 
+  is misuse or abuse.
+how_to_implement: 'You must install the AWS App for Splunk (version 5.1.0 or later)
+  and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
+  inputs.\
+  
+  In addition, you must have the Machine Learning Toolkit (MLTK) version
+  >= 4.2 installed, along with any required dependencies. Depending on the number
+  of users in your environment, you may also need to adjust the value for max_inputs
+  in the MLTK settings for the DensityFunction algorithm, then ensure that the search
+  completes in a reasonable timeframe. By default, the search builds the model using
+  the past 30 days of data. You can modify the search window to build the model over
+  a longer period of time, which may give you better results. You may also want to
+  periodically re-run this search to rebuild the model with the latest data.\
+  
+  More information on the algorithm used in the search can be found at 
+  `https://docs.splunk.com/Documentation/MLApp/4.2.0/User/Algorithms#DensityFunction`.'
+id: b28ed6de-e4ba-40f7-ae0a-93a088c774ab
+known_false_positives: ''
+maintainers:
+  - company: Splunk
+    email: [email protected]
+    name: Jason Brewer
+modification_date: '2019-11-14'
+name: Baseline of Excessive AWS Instances Terminated by User - MLTK
+original_authors:
+  - company: Splunk
+    email: [email protected]
+    name: Jason Brewer
+spec_version: 2
+type: splunk
+version: '1.0'

detections/ec2_excessive_terminateinstances_mltk.yml

@@ -0,0 +1,107 @@
+asset_type: AWS Instance
+baselines:
+  - id: b28ed6de-e4ba-40f7-ae0a-93a088c774ab
+    name: Baseline of Excessive AWS Instances Terminated by User - MLTK
+    type: splunk
+confidence: medium
+creation_date: '2019-11-14'
+data_metadata:
+  data_source:
+    - AWS CloudTrail logs
+  data_sourcetypes:
+    - aws:cloudtrail
+  providing_technologies:
+    - AWS
+description: This search looks for CloudTrail events where a user successfully terminates
+  an abnormally high number of instances.
+detect:
+  splunk:
+    correlation_rule:
+      macros:
+        - ec2_excessive_terminateinstances_mltk_input_filter
+      notable:
+        nes_fields: src_user
+        rule_description: An abnormally high number of instances were terminated by
+          a user within in a 10-minute window
+        rule_title: High Number of instances terminated by $src_user$
+      risk:
+        risk_object: src_user
+        risk_object_type:
+          - user
+        risk_score: 30
+      schedule:
+        cron_schedule: '*/10 * * * *'
+        earliest_time: -70m@m
+        latest_time: -10m@m
+      search: >-
+        sourcetype=aws:cloudtrail eventName=TerminateInstances errorCode=success `ec2_excessive_terminateinstances_mltk_input_filter`
+        | bucket span=10m _time 
+        | stats count as instances_terminated by _time src_user 
+        | apply ec2_excessive_terminateinstances_v1 
+        | rename "IsOutlier(instances_terminated)" as isOutlier 
+        | where isOutlier=1
+      suppress:
+        suppress_fields: src_user
+        suppress_period: 3600s
+eli5: In this search, we query CloudTrail logs to look for events where an instance
+  is successfully terminated by a particular user. Since we want to detect a high number
+  of instances terminated within a short period, we create event buckets for 10-minute
+  windows. We then compare the total number of instances terminated by a particular
+  user against the saved baseline data in the model ec2_excessive_terminateinstances_v1.
+entities:
+  - src_user
+how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
+  and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
+  inputs. The threshold value should be tuned to your environment.
+id: 1c02b86a-cd85-473e-a50b-014a9ac8fe3e
+investigations:
+  - id: bc91a8cd-35e7-4bb2-6140-e756cc46fd71
+    name: AWS Investigate User Activities By ARN
+    type: splunk
+  - id: 0e40fe83-3edb-4d86-8206-8fed36529ca6
+    name: Get EC2 Launch Details
+    type: splunk
+  - id: f3db4d1b-5f33-4b01-c541-c7ah9514c242
+    name: Get EC2 Instance Details by instanceId
+    type: splunk
+  - id: 3d6c3213-5fff-4a1e-b57d-b24c262171e7
+    name: Get Notable History
+    type: splunk
+  - id: f3fb4d1b-5f33-4b01-b541-c7af9534c242
+    name: Get Notable Info
+    type: splunk
+  - id: bc91a8cf-35e7-4bb2-8140-e756cc06fd74
+    name: Get User Information from Identity Table
+    type: splunk
+  - id: bc91a8cd-35e7-4bb2-6140-e756cc46fd11
+    name: Investigate AWS activities via region name
+    type: splunk
+known_false_positives: Many service accounts configured within an AWS infrastructure
+  are known to exhibit this behavior. Please adjust the threshold values and filter
+  out service accounts from the output. Always verify if this search alerted on a
+  human user.
+maintainers:
+  - company: Splunk
+    email: [email protected]
+    name: Jason Brewer
+mappings:
+  cis20:
+    - CIS 13
+  kill_chain_phases:
+    - Actions on Objectives
+  mitre_attack:
+    - Execution
+  nist:
+    - DE.DP
+    - DE.AE
+modification_date: '2019-11-14'
+name: Abnormally High AWS Instances Terminated by User - MLTK
+original_authors:
+  - company: Splunk
+    email: [email protected]
+    name: Jason Brewer
+references: []
+security_domain: network
+spec_version: 2
+type: splunk
+version: '1.0'

macros/ec2_excessive_runinstances_mltk_input_filter.yml

@@ -1,3 +1,3 @@
 description: Use this macro to add additional filters for monitoring ec2 
-  runinstances [eg - userName != 'someUserNameExperiencingFalsePositives'].
+  runinstances [eg - src_user != 'someUserNameExperiencingFalsePositives'].
 name: ec2_excessive_runinstances_mltk_input_filter

macros/ec2_excessive_terminateinstances_mltk_input_filter.yml

@@ -0,0 +1,3 @@
+description: Use this macro to add additional filters for monitoring ec2 
+  terminateinstances [eg - src_user != 'someUserNameExperiencingFalsePositives'].
+name: ec2_excessive_terminateinstances_mltk_input_filter

stories/aws_ec2.yml

@@ -23,6 +23,9 @@ detections:
   - detection_id: dec41ad5-d579-42cb-b4c6-f5dbb778bbe5
     name: Abnormally High AWS Instances Launched by User - MLTK
     type: splunk
+  - detection_id: 1c02b86a-cd85-473e-a50b-014a9ac8fe3e
+    name: Abnormally High AWS Instances Terminated by User - MLTK
+    type: splunk
 id: 2e8948a5-5239-406b-b56b-6c50f1268af3
 maintainers:
   - company: Splunk