Skip to content

2.0.0
Compare
Choose a tag to compare
released this 22 May 13:10
· 2870 commits to develop since this release
2.0.0
14bde43

Installation instructions:
https://github.com/certtools/intelmq/blob/2.0.0/docs/INSTALL.md
Upgrade instructions:
https://github.com/certtools/intelmq/blob/2.0.0/docs/UPGRADING.md

There are some features considered as beta and marked as such in the documentation, do not use them in production yet.

See also the changelog for 2.0.0.beta1 below.

Configurations

  • Defaults: New parameters statistics_host, statistics_port, statistics_databasae, statistics_password for statistics redis database (#1402).

Core

  • Add more and fix some existing type annotations.
  • intelmq.lib.bot:
    • Use statistics_* parameters for bot's statistics (#1402).
    • Introduce collector_empty_process for collectors with an empty process() method, hardcoded 1s minimum sleep time, preventing endless loops, causing high load (#1364).
    • Allow to disable multithreading by initialization parameter, used by intelmqctl / the bot debugger (#1403).
  • intelmq.lib.pipeline: redis: OOM can also be low memory, add this to log message (#1405).
  • intelmq.lib.harmonization: ClassificationType: Update RSIT mapping (#1380):
    • replace botnet drone with infected-system
    • replace infected system with infected-system
    • replace ids alert with ids-alert
    • replace c&c with c2server
    • replace malware configuration with malware-configuration
    • sanitize replaces these values on the fly
  • Allow using non-opt/ (LSB) paths with environment variable INTELMQ_PATHS_NO_OPT.
  • Disable/disallow threading for all collectors and some other bots.

Development

  • Applied isort to all core files and core-related test files, sorting the imports there (every thing except bots and bots' tests).

Harmonization

  • See the Core section for the changes in the allowed values for classification.type.

Bots

  • Use the new RSIT types in several types, see above

Parsers

  • intelmq.bots.parsers.spamhaus.parser_cert: Added support for extortion events.

Experts

  • added intelmq.bots.experts.do_portal.expert.

Outputs

  • intelmq.bots.outputs.elasticsearch.output: Support for TLS added (#1406).
  • intelmq.bots.outputs.tcp.output: Support non-intelmq counterparts again. New parameter counterpart_is_intelmq, see NEWS.md for more information (#1385).

Packaging

  • Update IntelMQ path fix patch after INTELMQ_PATHS_NO_OPT introduction, provide INTELMQ_PATHS_OPT environment variable for packaged instances.

Tests

  • test_conf: For yaml use safe_load instead of unsafe load.
  • Travis: Switch distribution from trusty to xenial, adapt scripts.
    • Add Python 3.7 to tests.
  • Don't use Cerberus 1.3 because of pyeve/cerberus#489

Tools

  • intelmqdump: Fix creation of pipeline object by providing a logger.
  • intelmqctl: Disable multithreading for interactive runs / the bot debugger (#1403).

Known issues

  • tests: capture logging with context manager (#1342)
  • pymongo 3.0 deprecates used insert method (#1063)
  • pymongo >= 3.5: authentication changes (#1062)
  • Bots started with IntelMQ-Manager stop when the webserver is restarted. (#952)
  • n6 parser: mapping is modified within each run (#905)
  • reverse DNS: Only first record is used (#877)
  • Corrupt dump files when interrupted during writing (#870)
Assets 2
Loading