Skip to content

Commit

Permalink
Merge branch 'main' into docs-filebeat-takeover
Browse files Browse the repository at this point in the history
  • Loading branch information
@VihasMakwana
VihasMakwana authored Jun 12, 2024
2 parents c45190f + 328670b commit 27f85b6
Show file tree
Hide file tree
Showing 97 changed files with 2,327 additions and 1,618 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG-developer.next.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,8 @@ The list below covers the major changes between 7.0.0-rc2 and main only.
- Fix copy arguments for strict aligned architectures. {pull}36976[36976]
- Fix panic when more than 32767 pipeline clients are active. {issue}38197[38197] {pull}38556[38556]
- Skip flakey metrics test on windows in filebeat httpjson input. {issue}39676[39676] {pull}39678[39678]
- Fix flakey test on Windows 2022 in packetbeat/route. {issue}39698[39698] {pull}39822[39822]
- Fix bug in minimum length for request trace logging. {pull}39834[39834]

==== Added

Expand Down Expand Up @@ -190,6 +192,7 @@ The list below covers the major changes between 7.0.0-rc2 and main only.
- Add Active Directory entity collector for Filebeat entity analytics. {pull}37854[37854]
- Make logs for empty and small files less noisy when using fingerprint file identity in filestream. {pull}38421[38421]
- Improve robustness and error reporting from packetbeat default route testing. {pull}39757[39757]
- Move x-pack/filebeat/input/salesforce jwt import to v5. {pull}39823[39823]

==== Deprecated

Expand Down
125 changes: 125 additions & 0 deletions CHANGELOG.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,131 @@
:issue: https://github.com/elastic/beats/issues/
:pull: https://github.com/elastic/beats/pull/

[[release-notes-8.14.0]]
=== Beats version 8.14.0
https://github.com/elastic/beats/compare/v8.13.4\...v8.14.0[View commits]

==== Breaking changes

*Filebeat*

- Removed deprecated ZScaler from Beats. Use the https://docs.elastic.co/integrations/zscaler_zia[Zscaler Internet Access] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Tomcat from Beats. Use the https://docs.elastic.co/integrations/apache_tomcat[Apache Tomcat] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Squid from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated SonicWall from Beats. Use the https://docs.elastic.co/integrations/sonicwall[SonicWall Firewall] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Snort from Beats. Use the https://docs.elastic.co/integrations/snort[Snort] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Radware from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Proofpoint from Beats. Use the https://docs.elastic.co/integrations/proofpoint_tap[Proofpoint TAP] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Netscout from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Microsoft DHCP from Beats. Use the https://docs.elastic.co/integrations/microsoft_dhcp[Microsoft DHCP] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Juniper Junos from Beats. Use the https://docs.elastic.co/integrations/juniper_srx[Juniper SRX] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Juniper Netscreen from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Infoblox from Beats. Use the https://docs.elastic.co/integrations/infoblox_nios[Infoblox NIOS] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Impreva from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Fortinet Client Endpoint from Beats. Use the https://docs.elastic.co/integrations/fortinet_forticlient[Fortinet FortiClient Logs] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Fortinet Fortimail from Beats. Use the https://docs.elastic.co/integrations/fortinet_fortimail[Fortinet FortiMail] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Fortinet Fortimanager from Beats. Use the https://docs.elastic.co/integrations/fortinet_fortimanager[Fortinet FortiManager Logs] Elastic integration instead. {pull}38037[38037]
- Removed deprecated F5 from Beats. Use the https://docs.elastic.co/integrations/f5_bigip[F5 BIG-IP] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Cylance from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Cisco Meraki from Beats. Use the https://docs.elastic.co/integrations/cisco_meraki[Cisco Meraki] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Cisco Nexus from Beats. Use the https://docs.elastic.co/integrations/cisco_nexus[Cisco Nexus] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Bluecoat from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Barracuda from Beats. Use the https://docs.elastic.co/integrations/barracuda[Barracuda Web Application Firewall] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Sophos UTM from Beats. Use the https://docs.elastic.co/integrations/sophos[Sophos] Elastic integration instead. {pull}38037[38037]
- Introduce input/netmetrics and refactor netflow input metrics. {pull}38055[38055]
- Update Salesforce module to use new Salesforce input. {pull}37509[37509]

*Heartbeat*

- Fix monitor state loader to not wait extra seconds for the last attempt. {pull}39621[39621]

==== Bugfixes

*Auditbeat*
- Set field types to correctly match ECS in sessionmd processor. {issue}38955[38955] {pull}38994[38994]
- Fix failing to enrich process events in sessionmd processor. {issue}38955[38955] {pull}39173[39173] {pull}39243[39243]
- Fix seccomp policy of FIM kprobes backend on arm64. {pull}39759[39759]

*Filebeat*
- Fix handling of endpoint for custom domains and ensure region, default_region, and region parsed from queue_url are applied in the order specified in the documentation for the awss3 input. {pull}39709[39709]
- Prevent HTTPJSON holding response bodies between executions. {issue}35219[35219] {pull}38116[38116]
- Fix the incorrect values generated by the uri_parts processor. {pull}38216[38216]
- Rename `activity_guid` to `activity_id` in ETW input events to suit other Windows inputs. {pull}38530[38530]
- Add missing provider registration and fix published entity for Active Directory entityanalytics provider. {pull}38645[38645]
- Fix handling of un-parsed JSON in O365 module. {issue}37800[37800] {pull}38709[38709]
- Fix filestream's registry GC: registry entries are now removed from the in-memory and disk store when they're older than the set TTL. {issue}36761[36761] {pull}38488[38488]
- Fix handling of truncated files in Filestream {issue}38070[38070] {pull}38416[38416]
- Fix panic when more than 32767 pipeline clients are active. {issue}38197[38197] {pull}38556[38556]
- Fix a bug in CloudWatch task allocation that could skip some logs. {issue}38918[38918] {pull}38953[38953]
- Prevent GCP Pub/Sub input blockage by increasing default value of `max_outstanding_messages`. {issue}35029[35029] {pull}38985[38985]
- entity-analytics input: Improve structured logging. {pull}38990[38990]
- Upgrade `azure-event-hubs-go` and `azure-storage-blob-go` dependencies. {pull}38861[38861]
- Fix concurrency/error handling bugs in the AWS S3 input that could drop data and prevent ingestion of large buckets. {pull}39131[39131]
- Fix EntraID query handling. {issue}39419[39419] {pull}39420[39420]
- Expand ID patterns in request trace logger for HTTP Endpoint. {pull}39656[39656]

*Heartbeat*

- Redact synthexec cmd output. {pull}39535[39535]

*Metricbeat*

- RabbitMQ/queue - Change the mapping type of `rabbitmq.queue.consumers.utilisation.pct` to `scaled_float` from `long` because the values fall within the range of `[0.0, 1.0]`. Previously, conversion to integer resulted in reporting either `0` or `1`.
- Fix timeout caused by the retrival of which indices are hidden. {pull}39165[39165]

*Winlogbeat*

- Fix error handling in perfmon metrics. {issue}38140[38140] {pull}39404[39404]

==== Added

*Affecting all Beats*

- Update Go version to 1.21.10. {pull}39467[39467]
- Enable early event encoding in the Elasticsearch output, improving CPU and memory use. {pull}38572[38572]

*Auditbeat*

- Add `add_session_metadata` processor, which enables session viewer on Auditbeat data. {pull}37640[37640]
- Add procfs backend to the `add_session_metadata` processor. {pull}38799[38799]
- Add `process.entity_id`, `process.group.name` and `process.group.id` in `add_process_metadata` processor. Make FIM module with Kprobes backend to always add an appropriately configured `add_process_metadata` processor to enrich file events. {pull}38776[38776]

*Filebeat*

- Add Saved Object name field to Kibana audit logs. {pull}38307[38307]
- Add Salesforce input. {pull}37331[37331]
- Add logging for cache processor file reads and writes. {pull}38052[38052]
- Support VPC endpoint for aws-s3 input SQS queue url. {pull}38189[38189]
- Add support for complex event objects in the HTTP Endpoint input. {issue}37910[37910] {pull}38193[38193]
- Parse more fields from Elasticsearch slowlogs. {pull}38295[38295]
- Update CEL mito extensions to v1.10.0 to add keys/values helper. {pull}38504[38504]
- Add support for Active Directory an entity analytics provider. {pull}37919[37919]
- Add AWS AWSHealth metricset. {pull}38370[38370]
- Add debugging breadcrumb to logs when writing request trace log. {pull}38636[38636]
- Add benchmark input and discard output. {pull}37437[37437]

*Libbeat*

- Add support for Linux capabilities in `add_process_metadata`. {pull}38252[38252]

*Metricbeat*

- Add support for `shards_stats.total_count` in Elasticsearch Monitoring data. {pull}38891[38891]
- Add SSL support to MySQL module. {pull}37997[37997]
- Add SSL support for Aerospike module. {pull}38126[38126]

*Winlogbeat*

- Use fixed size buffer at first pass for event parsing, improving throughput. {issue}39530[39530] {pull}39544[39544]

==== Deprecated

*Filebeat*

- Deprecate `syslog` input in favor of `syslog` processor. {issue}37555[37555] {pull}38277[38277]
- Deprecate `o365audit` input in favor of `CEL` input. {issue}37719[37719] {pull}38922[38922]


[[release-notes-8.13.4]]
=== Beats version 8.13.4
https://github.com/elastic/beats/compare/v8.13.3\...v8.13.4[View commits]
Expand Down
30 changes: 10 additions & 20 deletions CHANGELOG.next.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,11 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Removed deprecated Sophos UTM from Beats. Use the https://docs.elastic.co/integrations/sophos[Sophos] Elastic integration instead. {pull}38037[38037]
- Introduce input/netmetrics and refactor netflow input metrics {pull}38055[38055]
- Update Salesforce module to use new Salesforce input. {pull}37509[37509]
- Tag events that come from a filestream in "take over" mode. {pull}39828[39828]
- Fix high IO and handling of a corrupted registry log file. {pull}35893[35893]

*Heartbeat*

- Fix monitor state loader to not wait extra seconds for the last attempt {pull}39621[39621]

*Metricbeat*

Expand Down Expand Up @@ -86,8 +86,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Fix panic when MaxRetryInterval is specified, but RetryInterval is not {pull}35820[35820]
- Support build of projects outside of beats directory {pull}36126[36126]
- Support Elastic Agent control protocol chunking support {pull}37343[37343]
- Upgrade elastic-agent-libs to v0.7.5. Removes obsolete "Treating the CommonName field on X.509 certificates as a host name..." deprecation warning for 8.0. {pull}37755[37755]
- aws: Add credential caching for `AssumeRole` session tokens. {issue}37787[37787]
- Lower logging level to debug when attempting to configure beats with unknown fields from autodiscovered events/environments {pull}[37816][37816]
- Set timeout of 1 minute for FQDN requests {pull}37756[37756]
- Fix the paths in the .cmd script added to the path by the Windows MSI to point to the new C:\Program Files installation location. https://github.com/elastic/elastic-stack-installers/pull/238
Expand All @@ -98,12 +96,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Rename the field "apache2.module.error" to "apache.module.error" in Apache error visualization. {issue}39480[39480] {pull}39481[39481]

*Auditbeat*
- Set field types to correctly match ECS in sessionmd processor {issue}38955[38955] {pull}38994[38994]
- Fix failing to enrich process events in sessionmd processor {issue}38955[38955] {pull}39173[39173] {pull}39243[39243]
- Prevent scenario of losing children-related file events in a directory for recursive fsnotify backend of auditbeat file integrity module {pull}39133[39133]
- Allow extra syscalls by auditbeat required in FIM with kprobes back-end {pull}39361[39361]
- Fix losing events in FIM for OS X by allowing always to walk an added directory to monitor {pull}39362[39362]
- Fix seccomp policy of FIM kprobes backend on arm64 {pull}39759[39759]



Expand All @@ -121,9 +113,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Fix handling of Juniper SRX structured data when there is no leading junos element. {issue}36270[36270] {pull}36308[36308]
- Fix Filebeat Cisco module with missing escape character {issue}36325[36325] {pull}36326[36326]
- Added a fix for Crowdstrike pipeline handling process arrays {pull}36496[36496]
- Fix m365_defender cursor value and query building. {pull}37116[37116]
- Fix TCP/UDP metric queue length parsing base. {pull}37714[37714]
- Update github.com/lestrrat-go/jwx dependency. {pull}37799[37799]
- [threatintel] MISP pagination fixes {pull}37898[37898]
- Fix file handle leak when handling errors in filestream {pull}37973[37973]
- Fix a race condition that could crash Filebeat with a "negative WaitGroup counter" error {pull}38094[38094]
Expand Down Expand Up @@ -161,6 +150,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Adjust State loader to only retry when response code status is 5xx {pull}37981[37981]
- Reset prctl dumpable flag after cap drop. {pull}38269[38269]
- Redact synthexec cmd output. {pull}39535[39535]
- Fix import of browser plugin for agentbeat. {pull}39818[39818]

*Heartbeat*

Expand All @@ -186,7 +176,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]

*Winlogbeat*

- Fix error handling in perfmon metrics. {issue}38140[38140] {pull}39404[39404]

*Elastic Logging Plugin*

Expand Down Expand Up @@ -226,10 +215,11 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
*Auditbeat*


*Auditbeat*


*Filebeat*

- Adding Saved Object name field to Kibana audit logs {pull}38307[38307]
- Update SQL input documentation regarding Oracle DSNs {pull}37590[37590]
- add documentation for decode_xml_wineventlog processor field mappings. {pull}32456[32456]
- httpjson input: Add request tracing logger. {issue}32402[32402] {pull}32412[32412]
- Add cloudflare R2 to provider list in AWS S3 input. {pull}32620[32620]
Expand Down Expand Up @@ -289,13 +279,14 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Make HTTP Endpoint input GA. {issue}38979[38979] {pull}39410[39410]
- Update CEL mito extensions to v1.12.2. {pull}39755[39755]
- Add support for base64-encoded HMAC headers to HTTP Endpoint. {pull}39655[39655]
- Add user group membership support to Okta entity analytics provider. {issue}39814[39814] {pull}39815[39815]
- Add request trace support for Okta and EntraID entity analytics providers. {pull}39821[39821]

*Auditbeat*


*Libbeat*

- Add support for linux capabilities in add_process_metadata. {pull}38252[38252]


*Heartbeat*
Expand All @@ -304,8 +295,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]

*Metricbeat*

- Add support for shards_stats.total_count in Elasticsearch Monitoring data. {pull}38891[38891]
- Add new fields to configure the lease duration, retry and renew when using leader elector with kubernetes autodiscover.{pull}38471[38471]
- Add per-thread metrics to system_summary {pull}33614[33614]
- Add GCP CloudSQL metadata {pull}33066[33066]
- Add GCP Carbon Footprint metricbeat data {pull}34820[34820]
Expand Down Expand Up @@ -348,8 +337,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]

*Filebeat*

- Deprecate `syslog` input in favor of `syslog` processor. {issue}37555[37555] {pull}38277[38277]
- Deprecate `o365audit` input in favor of `CEL` input. {issue}37719[37719] {pull}38922[38922]

*Heartbeat*

Expand Down Expand Up @@ -430,6 +417,9 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]









Expand Down
Loading

0 comments on commit 27f85b6

Please sign in to comment.