Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added RST IoC Lookup connector. Fixes for Report Hub and Threat Feed #2864

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

k1r10n
Copy link
Contributor

@k1r10n k1r10n commented Oct 29, 2024

Proposed changes

  • A new RST IoC Lookup Connector: cost-effective and functional enrichment connector for observables and indicators
  • New parameters to give options for users to disable/enable generation of 'related-to' and observables when using the RST Report Hub connector
  • Minor fixes to RST Threat Feed connector: indicator patterns standardised, only_new logic changed, code reformatting, description are nulled for high level objects as intrusion-set, malware, tools, etc profiles are to come from RST Threat Library

Related issues

Checklist

  • I consider the submitted work as finished
  • I tested the code for its functionality using different use cases
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

Copy link
Member

@Powlinett Powlinett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @k1r10n , thank you for your PR and this new connector!

As I wrote in the comments, there is an issue with CONNECTOR_UPDATE_EXISTING_DATA env var:

  • rst-threat-hub doesn't ingest anything as VALIDATION_ERROR is raised for each message
  • rst-threat-feed ingests data and I didn't see any VALIDATION_ERROR so far (there are many MISSING_REFERENCE_ERROR but it's already the case on master, your fixes/updates are not responsible)
  • I tested rst-ioc-lookup manually and in auto mode, it seems to working as intended in both ways

FIY, all my tests have been done by running connectors locally with env vars set in config.yml.

Could you fix the issue in the comments so we can merge your PR please? Thanks 😇

}
self.update_existing_data = get_config_variable(
"CONNECTOR_UPDATE_EXISTING_DATA",
["connector", "update_existing_data"],
config,
True,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it does what it's supposed to do. True here refers to isNumber argument (which parse an environment variable to an integer), but I think self.update_existing_data needs to be a boolean.
If not, a VALIDATION_ERROR is raised during ingestion.

"CONNECTOR_UPDATE_EXISTING_DATA",
["connector", "update_existing_data"],
config,
True,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as in rst-report-hub connector

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it because the variable CONNECTOR_UPDATE_EXISTING_DATA was deprecated? Should we remove it as now merging is done using confidence levels?

@Powlinett Powlinett added the community use to identify PR from community label Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community use to identify PR from community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants