Skip to content

Latest commit

 

History

History
402 lines (300 loc) · 23.1 KB

CHANGELOG.next.asciidoc

File metadata and controls

402 lines (300 loc) · 23.1 KB
Raw

Beats version HEAD

Breaking changes

Affecting all Beats - Fix status reporting to Elastic-Agent when output configuration is invalid running under Elastic-Agent 35719

Auditbeat

Filebeat

Heartbeat

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

  • Add "event.category" and "event.type" to Sysmon module for EventIDs 8, 9, 19, 20, 27, 28, 255 35193

Functionbeat

Elastic Logging Plugin

Bugfixes

Affecting all Beats - Support for multiline zookeeper logs 2496 - Allow clock_nanosleep in the default seccomp profiles for amd64 and 386. Newer versions of glibc (e.g. 2.31) require it. 33792 - Disable lockfile when running under elastic-agent. 33988 - Fix lockfile logic, retry locking 34194 - Add checks to ensure reloading of units if the configuration actually changed. 34346 - Fix namespacing on self-monitoring 32336 - Fix race condition when stopping runners 32433 - Fix concurrent map writes when system/process code called from reporter code 32491 - Log errors from the Elastic Agent V2 client errors channel. Avoids blocking when error occurs communicating with the Elastic Agent. 34392 - Only log publish event messages in trace log level under elastic-agent. 34391 - Fix issue where updating a single Elastic Agent configuration unit results in other units being turned off. 34504 - Fix dropped events when monitor a beat under the agent and send its Host info log entry. 34599 - Fix namespacing on self-monitoring 32336 - Fix race condition when stopping runners 32433 - Fix concurrent map writes when system/process code called from reporter code 32491 - Fix panics when a processor is closed twice 34647 - Update elastic-agent-system-metrics to v0.4.6 to allow builds on mips platforms. 34674 - The Elasticsearch output now splits large requests instead of dropping them when it receives a StatusRequestEntityTooLarge error. 34911 - Fix Beats started by agent do not respect the allow_older_versions: true configuration flag 34227 34964 - Fix performance issues when we have a lot of inputs starting and stopping by allowing to disable global processors under fleet. 35000 35031 - In cases where the matcher detects a non-string type in a match statement, report the error as a debug statement, and not a warning statement. 35119 - 'add_cloud_metadata' processor - add cloud.region field for GCE cloud provider - 'add_cloud_metadata' processor - update azure metadata api version to get missing cloud.account.id field - Make sure k8s watchers are closed when closing k8s meta processor. 35630 - Upgraded apache arrow library used in x-pack/libbeat/reader/parquet from v11 to v12.0.1 in order to fix cross-compilation issues 35640

Auditbeat

Filebeat

  • [Auditbeat System Package] Added support for Apple Silicon chips. 34433

  • [Azure blob storage] Changed logger field name from container to container_name so that it does not clash with the ecs field name container. 34403

  • [GCS] Added support for more mime types & introduced offset tracking via cursor state. Also added support for automatic splitting at root level, if root level element is an array. 34155

  • [httpsjon] Improved error handling during pagination with chaining & split processor 34127

  • [Azure blob storage] Added support for more mime types & introduced offset tracking via cursor state. 33981

  • Fix EOF on single line not producing any event. 30436 33568

  • Fix handling of error in states in direct aws-s3 listing input 33513 33722

  • Fix httpjson input page number initialization and documentation. 33400

  • Add handling of AAA operations for Cisco ASA module. 32257 32789

  • Fix gc.log always shipped even if gc fileset is disabled 30995

  • Fix handling of empty array in httpjson input. 32001

  • Fix reporting of filebeat.events.active in log events such that the current value is always reported instead of the difference from the last value. 33597

  • Fix splitting array of strings/arrays in httpjson input 30345 33609

  • Fix Google workspace pagination and document ID generation. 33666

  • Fix PANW handling of messages with event.original already set. 33829 33830

  • Rename identity as identity_name when the value is a string in Azure Platform Logs. 33654

  • Fix 'requires pointer' error while getting cursor metadata. 33956

  • Fix input cancellation handling when HTTP client does not support contexts. 33962 33968

  • Update mito CEL extension library to v0.0.0-20221207004749-2f0f2875e464 33974

  • Fix CEL result deserialisation when evaluation fails. 33992 33996

  • Fix handling of non-200/non-429 status codes. 33999 34002

  • [azure-eventhub input] Switch the run EPH run mode to non-blocking 34075

  • [google_workspace] Fix pagination and cursor value update. 34274

  • Fix handling of quoted values in auditd module. 22587 34069

  • Fixing system tests not returning expected content encoding for azure blob storage input. 34412

  • [Azure Logs] Fix authentication_processing_details parsing in sign-in logs. 34330 34478

  • Prevent Elasticsearch from spewing log warnings about redundant wildcard when setting up ingest pipelines. 34249 34550

  • Gracefully handle Windows event channel not found errors in winlog input. 30201 34605

  • Fix the issue of cometd input worker getting closed in case of a network connection issue and an EOF error. 34326 34327

  • Fix for httpjson first_response object throwing false positive errors by making it a flag based object 34747 34748

  • Fix errors and panics due to re-used processors 34761

  • Add missing Basic Authentication support to CEL input 34609 34689

  • [Gcs Input] - Added missing locks for safe concurrency 34914

  • Fix the ignore_inactive option being ignored in Filebeat’s filestream input 34770

  • Fix TestMultiEventForEOFRetryHandlerInput unit test of CometD input 34903

  • Add input instance id to request trace filename for httpjson and cel inputs 35024

  • Fix panic in TCP and UDP inputs on Linux when collecting socket metrics from OS. 35064

  • Correctly collect TCP and UDP metrics for unspecified address values. 35111

  • Fix base for UDP and TCP queue metrics and UDP drops metric. 35123

  • Sanitize filenames for request tracer in httpjson input. 35143

  • decode_cef processor: Fix ECS output by making observer.ip into an array of strings instead of string. 35140 35149

  • Fix handling of MySQL audit logs with strict JSON parser. 35158 35160

  • Sanitize filenames for request tracer in cel input. 35154

  • Fix accidental error overwrite in defer statement in entityanalytics Azure AD input. 35153 35169

  • Fixing the grok expression outputs of log files 35221

  • Fixes "Can only start an input when all related states are finished" error when running under Elastic-Agent 35250 33653

  • Move repeated Windows event channel not found errors in winlog input to debug level. 35314 35317

  • Fix crash when processing forwarded logs missing a message. 34705 34865

  • Fix crash when loading azurewebstorage cursor with no partially processed data. 35433

  • Add support in s3 input for JSON with array of objects. 35475

  • RFC5424 syslog timestamps with offset 'Z' will be treated as UTC rather than using the default timezone. 35360

  • Fix syslog message parsing for fortinet.firewall to take into account quoted values. 35522

  • [system] sync system/auth dataset with system integration 1.29.0. 35581

  • [GCS Input] - Fixed an issue where bucket_timeout was being applied to the entire bucket poll interval and not individual bucket object read operations. Fixed a map write concurrency issue arising from data races when using a high number of workers. Fixed the flaky tests that were present in the GCS test suit. 35605

  • Fix filestream false positive log error "filestream input with ID 'xyz' already exists" 31767

  • Fix error when trying to use include_message parser 35440

  • Fix handling of IPv6 unspecified addresses in TCP input. 35064 35637

  • Fixed a minor code error in the GCS input scheduler where a config value was being used directly instead of the source struct. 35729

  • Improve error reporting and fix IPv6 handling of TCP and UDP metric collection. 35772

  • Fix CEL input JSON marshalling of nested objects. 35763 35774

Heartbeat

  • Fix panics when parsing dereferencing invalid parsed url. 34702

  • Fix broken zip URL monitors. NOTE: Zip URL Monitors will be removed in version 8.7 and replaced with project monitors. 33723

  • Fix integration hashing to prevent reloading all when updated. 34697

  • Fix release of job limit semaphore when context is cancelled. 34697

  • Fix bug where states.duration_ms was incorrect type. 33563

  • Fix handling of long UDP messages in UDP input. 33836 33837

  • Fix browser monitor summary reporting as up when monitor is down. 33374 33819

  • Fix beat capabilities on Docker image. 33584

  • Fix serialization of state duration to avoid scientific notation. 34280

  • Enable nodejs engine strict validation when bundling synthetics. 34470 with the ecs field name container. 34403 automatic splitting at root level, if root level element is an array. 34155

  • Fix broken mapping for state.ends field. 34891

  • Fix issue using projects in airgapped environments by disabling npm audit. 34936

  • Fix broken state ID location naming. 35336

  • Fix project monitor temp directories permission to include group access. 35398

  • Fix output pipeline exit on run_once. 35376

  • Fix formatting issue with socket trace timeout. 35434

  • Update gval version. 35636

  • Fix serialization of processors when running diagnostics. 35698

Heartbeat

Heartbeat

Heartbeat

Auditbeat

Filebeat

Auditbeat

Filebeat

  • Sanitize filenames for request tracer in cel input. 35154

Heartbeat

Metricbeat

  • in module/windows/perfmon, changed collection method of the second counter value required to create a displayable value 32305

  • Fix and improve AWS metric period calculation to avoid zero-length intervals 32724

  • Add missing cluster metadata to k8s module metricsets 32979 33032

  • Add GCP CloudSQL region filter 32943

  • Fix logstash cgroup mappings 33131

  • Remove unused elasticsearch.node_stats.indices.bulk.avg_time.bytes mapping 33263

  • Fix kafka dashboard field names 33555

  • Add tags to events based on parsed identifier. 33472

  • Support Oracle-specific connection strings in SQL module 32089 32293

  • Remove deprecated metrics from controller manager, scheduler and proxy 34161

  • Fix metrics split through different events and metadata not matching for aws cloudwatch. 34483

  • Fix metadata enricher with correct container ids for pods with multiple containers in container metricset. Align kubernetes.container.id and container.id fields for state_container metricset. 34516

  • Make generic SQL GA 34637

  • Collect missing remote_cluster in elasticsearch ccr metricset 34957

  • Add context with timeout in AWS API calls 35425

  • Fix no error logs displayed in CloudWatch EC2, RDS and SQS metadata 34985 35035

  • Remove Beta warning from IIS application_pool metricset 35480

  • Improve documentation for ActiveMQ module 35113 35558

  • Resolve statsd module’s prematurely halting of metrics parsing upon encountering an invalid packet. 35075

Osquerybeat

  • Adds the elastic_file_analysis table to the Osquery extension for macOS builds. 35056

Packetbeat

  • Fix double channel close panic when reloading. 35324

  • Fix BPF filter setting not being applied to sniffers. 35363 35484

  • Fix handling of Npcap installation options from Fleet. 35541

Winlogbeat

  • Fix handling of event data with keys containing dots. 34345 34549

  • Gracefully handle channel not found errors. 30201 34605

  • Clarify query term limits warning and remove link to missing Microsoft doc page. 34715

  • Improve documentation for event_logs.name configuration. 34931

  • Move repeated channel not found errors to debug level. 35314 35317

  • Fix panic due to misrepresented buffer use. 35437

  • Prevent panic on closing iterators on empty channels in experimental API. 33966 35423

  • Allow program termination when attempting to open an absent channel. 35474

Functionbeat

Elastic Logging Plugin

Added

Affecting all Beats

  • Added append Processor which will append concrete values or values from a field to target. 29934 33364

  • Allow users to enable features via configuration, starting with the FQDN reporting feature. 1070 34456

  • Add Hetzner Cloud as a provider for add_cloud_metadata. 35456

  • Reload Beat when TLS certificates or key files are modified. 34408 34416

  • Upgrade version of elastic-agent-autodiscover to v0.6.1 for improved memory consumption on k8s. 35483

  • Added orchestrator.cluster.id and orchestrator.cluster.name fields to the add_cloud_metadata processor, AWS cloud provider. 35182

  • Lowercase reported hostnames per Elastic Common Schema (ECS) guidelines for the host.name field. Upgraded github.com/elastic/go-sysinfo to 1.11.0. 35652

Auditbeat

Filebeat

  • add documentation for decode_xml_wineventlog processor field mappings. 32456

  • httpjson input: Add request tracing logger. 32402 32412

  • Add cloudflare R2 to provider list in AWS S3 input. 32620

  • Add support for single string containing multiple relation-types in getRFC5988Link. 32811

  • Added separation of transform context object inside httpjson. Introduced new clause .parent_last_response.* 33499

  • Cloud Foundry input uses server-side filtering when retrieving logs. 33456

  • Add parse_aws_vpc_flow_log processor. 33656

  • Update aws.vpcflow dataset in AWS module have a configurable log format and to produce ECS 8.x fields. 33699

  • Modified aws-s3 input to reduce mutex contention when multiple SQS message are being processed concurrently. 33658

  • Disable "event normalization" processing for the aws-s3 input to reduce allocations. 33673

  • Add Common Expression Language input. 31233

  • Add support for http+unix and http+npipe schemes in httpjson input. 33571 33610

  • Add support for http+unix and http+npipe schemes in cel input. 33571 33712

  • Add decode_duration, move_fields processors. 31301

  • Add backup to bucket and delete functionality for the aws-s3 input. 30696 33559

  • Add metrics for UDP packet processing. 33870

  • Convert UDP input to v2 input. 33930

  • Improve collection of risk information from Okta debug data. 33677 34030

  • Adding filename details from zip to response for httpjson 33952 34044

  • Allow user configuration of keep-alive behaviour for HTTPJSON and CEL inputs. 33951 34014

  • Add support for polling system UDP stats for UDP input metrics. 34070

  • Add support for recognizing the log level in Elasticsearch JVM logs 34159

  • Add new Entity Analytics input with Azure Active Directory support. 34305

  • Added metric sqs_lag_time for aws-s3 input. 34306

  • Add metrics for TCP packet processing. 34333

  • Add metrics for unix socket packet processing. 34335

  • Add beta take over mode for filestream for simple migration from log inputs 34292

  • Add pagination support for Salesforce module. 34057 34065

  • Allow users to redact sensitive data from CEL input debug logs. 34302

  • Added support for HTTP destination override to Google Cloud Storage input. 34413

  • Added metric sqs_messages_waiting_gauge for aws-s3 input. 34488

  • Add support for new Rabbitmq timestamp format for logs 34211

  • Allow user configuration of timezone offset in Cisco ASA and FTD modules. 34436

  • Allow user configuration of timezone offset in Checkpoint module. 34472

  • Add support for Okta debug attributes, risk_reasons, risk_behaviors and factor. 33677 34508

  • Fill okta.request.ip_chain.* as a flattened object in Okta module. 34621

  • Fixed GCS log format issues. 34659

  • Add nginx.ingress_controller.upstream.ip to related.ip 34645 34672

  • Include NAT and firewall IPs in related.ip in Fortinet Firewall module. 34640 34673

  • Add Basic Authentication support on constructed requests to CEL input 34609 34689

  • Add string manipulation extensions to CEL input 34610 34689

  • Add unix socket log parsing for nginx ingress_controller 34732

  • Added metric sqs_worker_utilization for aws-s3 input. 34793

  • Improve CEL input documentation 34831

  • Add metrics documentation for CEL and AWS CloudWatch inputs. 34887 34889

  • Register MIME handlers for CSV types in CEL input. 34934

  • Add MySQL authentication message parsing and related.ip and related.user fields 34810

  • Mention mito CEL tool in CEL input docs. 34959

  • Add nginx ingress_controller parsing if one of upstreams fails to return response 34787

  • Allow neflow v9 and ipfix templates to be shared between source addresses. 35036

  • Add support for collecting IPv6 metrics. 35123

  • Add oracle authentication messages parsing 35127

  • Add sanitization capabilities to azure-eventhub input 34874

  • Add support for CRC validation in Filebeat’s HTTP endpoint input. 35204

  • Add support for CRC validation in Zoom module. 35604

  • Add execution budget to CEL input. 35409

  • Add XML decoding support to HTTPJSON. 34438 35235

  • Add delegated account support when using Google ADC in httpjson input. 35507

  • Add support for collecting httpjson metrics. 35392

  • Add XML decoding support to CEL. 34438 35372

  • Mark CEL input as GA. 35559

  • Add metrics for gcp-pubsub input. 35614

  • [GCS] Added scheduler debug logs and improved the context passing mechanism by removing them from struct params and passing them as function arguments. 35674

  • Allow non-AWS endpoints for awss3 input. 35496 35520

Auditbeat - Migration of system/package module storage from gob encoding to flatbuffer encoding in bolt db. 34817

Libbeat - Added support for apache parquet file reader. 34662 35183

Heartbeat - Users can now configure max scheduler job limits per monitor type via env var. 34307 - Added status to monitor run log report. - Removed beta label for browser monitors. 35424.

Metricbeat

  • Add per-thread metrics to system_summary 33614

  • Add GCP CloudSQL metadata 33066

  • Add GCP Redis metadata 33701

  • Remove GCP Compute metadata cache 33655

  • Add support for multiple regions in GCP 32964

  • Add GCP Redis regions support 33728

  • Add namespace metadata to all namespaced kubernetes resources. 33763

  • Changed cloudwatch module to call ListMetrics API only once per region, instead of per AWS namespace 34055

  • Add beta ingest_pipeline metricset to Elasticsearch module for ingest pipeline monitoring 34012

  • Handle duplicated TYPE line for prometheus metrics 18813 33865

  • Add GCP Carbon Footprint metricbeat data 34820

  • Add event loop utilization metric to Kibana module 35020

  • Support collecting metrics from both the monitoring account and linked accounts from AWS CloudWatch. 35540

  • Add new parameter include_linked_accounts to enable/disable metrics collection from multiple linked AWS Accounts 35648

  • Migrate Azure Billing, Monitor, and Storage metricsets to the newer SDK. 33585

  • Add support for float64 values parsing for statsd metrics of counter type. 35099

Osquerybeat

Packetbeat

  • Added packetbeat.interfaces.fanout_group to allow a Packetbeat sniffer to join an AF_PACKET fanout group. 35451 35453

  • Add AF_PACKET metrics. 35428 35489

Winlogbeat

Functionbeat

Winlogbeat

  • Set host.os.type and host.os.family to "windows" if not already set. 35435

  • Handle empty DNS answer data in QueryResults for the Sysmon Pipeline 35207

Elastic Log Driver Elastic Logging Plugin

Deprecated

Auditbeat

Filebeat

Heartbeat

  • Removed zip_url and local browser sources. 35429

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

Functionbeat

Elastic Logging Plugin

Known Issues