v4.4.1

Update CLI release_notes workflow for a bit more control on the branch we diff against to generate those notes. Previously, we could only diff against a tag.

What's Changed

• add --compare_against flag to release_notes action by @patel-bhavin in #311

Full Changelog: v4.4.0...v4.4.1

v4.4.0

Summary

contentctl 4.4.0 includes a significant number of fixes, updates, and new features.
Most notably, we now include support for

• Dashboard Objects - Dashboards can now be defined as content in the dashboards/ folder after creating a new app! These dashboards should be created in Splunk by creating a Simple XML Dashboard. Go to the "View Source" button when editing your dashboard to extract the JSON that represents that dashboard. Each dashboard is represented by a YML file and this JSON file (the JSON file should have the same name as the YML file. You can see some example dashboards that ESCU ships here: https://github.com/splunk/security_content/tree/develop/dashboards

• Drilldown Searches: Production searches which are NOT type: Hunting are now required to have two Drilldown searches. These now render in the Enterprise Security UI and make triaging and investigating your alerts much easier. For some example Drilldowns, please refer here: https://github.com/splunk/contentctl/blob/cfda377c6887e28e02bb1798382ac0070b7983c2/contentctl/templates/detections/endpoint/anomalous_usage_of_7zip.yml#L32-L40

• Throttling/Alert Suppression: In order to avoid too many alerts being generated in a given time frame, we have added support for Throttling/Alert Suppression on a per detection basis. Please refer to the inline documentation here for more information to:https://github.com/splunk/contentctl/blob/main/contentctl/objects/throttling.py . Splunk provides more information about throttling here: https://docs.splunk.com/Documentation/Splunk/9.3.1/Alert/ThrottleAlerts . An example throttling section of your Detection YML, under the "tags" section, looks like:

throttling:
  period: 3600s #time period to throttle
  fields: name,host # fields to throttle on

What's Changed

• Allow absent tests for experimental detections by @linuxdaemon in #36
• Update new content generator with new formats by @linuxdaemon in #44
• Handle stopped containers in testing by @linuxdaemon in #42
• Customer prs 1 by @pyth0n1c in #86
• Fix error on missing roles by @pyth0n1c in #190
• Add fields as requested by @pyth0n1c in #169
• Add UI dispatch app by @pyth0n1c in #145
• Update setuptools requirement from >=69.5.1,<71.0.0 to >=69.5.1,<72.0.0 by @dependabot in #196
• Handling when a user does not answer one of the questions by @yaleman in #189
• Update setuptools requirement from >=69.5.1,<71.0.0 to >=69.5.1,<72.0.0 by @dependabot in #202
• Update setuptools requirement from >=69.5.1,<71.0.0 to >=69.5.1,<72.0.0 by @dependabot in #205
• Handling the case where there are no tests by @yaleman in #198
• No tests fix by @pyth0n1c in #207
• Update setuptools requirement from >=69.5.1,<71.0.0 to >=69.5.1,<73.0.0 by @dependabot in #209
• Add Alert Suppression (throttling) support to detections by @pyth0n1c in #192
• Dashboard Support by @pyth0n1c in #147
• Fix name length by @pyth0n1c in #213
• improve output of risk severity field. by @pyth0n1c in #191
• contentctl v4.4.0 by @pyth0n1c in #179
• Ryanplasma add explanation by @pyth0n1c in #296
• Add type_list to annotations by @pyth0n1c in #293
• Fix datasource in contentctl new by @pyth0n1c in #297
• Optionally suppress missing detections during metadata validation by @pyth0n1c in #305
• Update xmltodict requirement from ^0.13.0 to >=0.13,<0.15 by @dependabot in #304
• Exception on malformatted unit tests in YMLs by @pyth0n1c in #300
• Refactoring for formatting and some logical error correction by @cmcginley-splunk in #308
• Mathieugonzales: replace deprecated pydantic validators by @pyth0n1c in #298
• Drilldown Support by @pyth0n1c in #256
• Allow testing with the default or custom_index by @ax-hsmith in #307
• Add more custom indexes by @pyth0n1c in #309

New Contributors

• @yaleman made their first contribution in #189
• @ax-hsmith made their first contribution in #307

Full Changelog: v4.3.5...v4.4.0

v4.3.5

In addition to some cleanup, this release includes two significant features:

1. Versioning enforcement has been added to that when a Detection is updated in a new release, its version field MUST be updated. This is important so that applications built with contentctl can take advantage of Splunk Enterprise Security 8's "Detection Versioning" feature! This enforcement has been added to the inspect workflow.
2. The enrichments workflow has changed, When building with enrichments, both the Atomic Red Team and Mitre CTI repos must be checked out. This update was made because it results in faster builds (when enrichments are enabled) and more stable and reliable builds using the Mitre CTI repo. We previously used the MITRE TAXII server, which is accessed via API in the attackcti client, but that API was frequently down, making us unable to build/test/release ESCU.

What's Changed

• Removal of more bits of SSA by @ljstella in #255
• Fix unintended whitespace by @pyth0n1c in #278
• Update bottle requirement from ^0.12.25 to >=0.12.25,<0.14.0 by @dependabot in #277
• Bareinit by @pyth0n1c in #288
• Update setuptools requirement from >=69.5.1,<75.0.0 to >=69.5.1,<76.0.0 by @dependabot in #290
• Feature: Adding version enforcement by @cmcginley-splunk in #280
• Require mitre/cti repo for enrichments by @pyth0n1c in #291

Full Changelog: v4.3.4...v4.3.5

v4.3.4

This PR includes extended support for ensuring that the appropriate Risk and Observable objects are created. See the PR linked below for more details.
There are also some small validation fixes around validating MITRE ID formats.

What's Changed

• Abstract Commonly Used Annotated Type Definitions by @pyth0n1c in #271
• Update setuptools requirement from >=69.5.1,<74.0.0 to >=69.5.1,<75.0.0 by @dependabot in #270
• Enabling risk/observable matching by @cmcginley-splunk in #241
• Update pyproject.toml by @pyth0n1c in #281

Full Changelog: v4.3.3...v4.3.4

v4.3.3

The action.correlationsearch.metadata field was updated to include an additional value called publish_date, a timestamp float representing when a detection was published.
Additionally, some cleanup was done around testing and the test_results/summary.yml was improved significantly to support better test results/tracking.
Finally, if searches use Baselines but have not been marked manual_test, they will throw runtime Exceptions during testing until Baselines are officially supported in the testing workflow.

What's Changed

• add publish_date field by @pyth0n1c in #239
• Responses to Comments by @pyth0n1c in #260
• Expanding coverage and other metrics in summary.yml by @cmcginley-splunk in #257

Full Changelog: v4.3.2...v4.3.3

v4.3.2

What's Changed

• add support for the entire mitre group metadata by @pyth0n1c in #253

Full Changelog: v4.3.1...v4.3.2

v4.3.1

Improve checking against observables. These changes ensure that Threat Objects and Risk Objects are created correctly.

What's Changed

• Threat objects by @ljstella in #234
• New observable role enum by @ljstella in #243
• Update setuptools requirement from >=69.5.1,<73.0.0 to >=69.5.1,<74.0.0 by @dependabot in #245

Full Changelog: v4.3.0...v4.3.1

v4.3.0

This change removes code and references to SSA as they are not applicable to external users.

What's Changed

• Update readme by @pyth0n1c in #244
• Remove SSA specific code by @P4T12ICK in #219

Full Changelog: v4.2.5...v4.3.0

v4.2.5

A number of small improvements from internal and community PRs. See the "What's Changed" below for details.

What's Changed

• Add a launcher to contentctl.py to allow easier debugging and launchi… by @Res260 in #212
• Update attackcti requirement from ^0.3.7 to >=0.3.7,<0.5.0 by @dependabot in #214
• Update on naming for the repo readme vs app readme by @pyth0n1c in #235
• Hotfix: Bumping integration testing timeout to compensate for recent bugfix by @cmcginley-splunk in #240

Full Changelog: v4.2.4...v4.2.5

v4.2.4

This change includes extended validation of the message: field of a detection when using --enable-integration-testing flag for contentctl test. This is mostly used for internal Splunk testing at this time.

It also now includes validation of DataSource Objects to ensure that the latest TA version is declared for each Data Source.

Finally, @Res260 made a contribution to get contentctl test working on Windows by fixing a path issue. Thanks!

What's Changed

• Update setuptools requirement from >=69.5.1,<71.0.0 to >=69.5.1,<73.0.0 by @dependabot in #215
• Tweaks to Data Source Validation by @pyth0n1c in #218
• Add latest TA version validation by @P4T12ICK in #216
• Allow contentctl test to work on Windows by fixing a path problem. by @Res260 in #217
• Addressed Casey's Feedback by @pyth0n1c in #222
• Adding risk message validation++ by @cmcginley-splunk in #92

New Contributors

• @Res260 made their first contribution in #217

Full Changelog: v4.2.2...v4.2.4