Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.x] Add known issue for 8.15.x memory increase. (backport #1403) #1405

Merged
merged 1 commit into from
Oct 23, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,31 @@ Also see:

Review important information about the {fleet} and {agent} 8.15.3 release.

[discrete]
[[known-issues-8.15.3]]
=== Known issues

[[known-issue-issue-41355-8.15.3]]
.The memory usage of {beats} based integrations is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage.
[%collapsible]
====

*Details*

In 8.15, events in the memory queue are not freed when they are acknowledged (as intended), but only when they are overwritten by later events in the queue buffer. This means for example if a configuration has a queue size of 5000, but the input data is low-volume and only 100 events are active at once, then the queue will gradually store more events until reaching 5000 in memory at once, then start replacing those with new events.

See {beats} issue link:https://github.com/elastic/beats/issues/40705[#40705].

*Impact* +

Memory usage may be higher than in previous releases depending on the throughput of {agent}. A fix is planned for 8.15.4.

- The worst memory increase is for low-throughput configs with large queues.
- For users whose queues were already sized proportionate to their throughput, memory use is increased but only marginally.
- Affected users can mitigate the higher memory usage by lowering their queue size.

====

[discrete]
[[security-updates-8.15.3]]
=== Security updates
Expand Down Expand Up @@ -64,6 +89,31 @@ Review important information about the {fleet} and {agent} 8.15.3 release.

Review important information about the {fleet} and {agent} 8.15.2 release.

[discrete]
[[known-issues-8.15.2]]
=== Known issues

[[known-issue-issue-41355-8.15.2]]
.The memory usage of {beats} based integrations is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage.
[%collapsible]
====

*Details*

In 8.15, events in the memory queue are not freed when they are acknowledged (as intended), but only when they are overwritten by later events in the queue buffer. This means for example if a configuration has a queue size of 5000, but the input data is low-volume and only 100 events are active at once, then the queue will gradually store more events until reaching 5000 in memory at once, then start replacing those with new events.

See {beats} issue link:https://github.com/elastic/beats/issues/40705[#40705].

*Impact* +

Memory usage may be higher than in previous releases depending on the throughput of {agent}. A fix is planned for 8.15.4.

- The worst memory increase is for low-throughput configs with large queues.
- For users whose queues were already sized proportionate to their throughput, memory use is increased but only marginally.
- Affected users can mitigate the higher memory usage by lowering their queue size.

====

[discrete]
[[enhancements-8.15.2]]
=== Enhancements
Expand Down Expand Up @@ -133,6 +183,27 @@ same package versions.

====

[[known-issue-issue-41355-8.15.1]]
.The memory usage of {beats} based integrations is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage.
[%collapsible]
====

*Details*

In 8.15, events in the memory queue are not freed when they are acknowledged (as intended), but only when they are overwritten by later events in the queue buffer. This means for example if a configuration has a queue size of 5000, but the input data is low-volume and only 100 events are active at once, then the queue will gradually store more events until reaching 5000 in memory at once, then start replacing those with new events.

See {beats} issue link:https://github.com/elastic/beats/issues/40705[#40705].

*Impact* +

Memory usage may be higher than in previous releases depending on the throughput of {agent}. A fix is planned for 8.15.4.

- The worst memory increase is for low-throughput configs with large queues.
- For users whose queues were already sized proportionate to their throughput, memory use is increased but only marginally.
- Affected users can mitigate the higher memory usage by lowering their queue size.

====

// end 8.15.1 relnotes

// begin 8.15.0 relnotes
Expand Down Expand Up @@ -171,6 +242,27 @@ If you're using {agent} on Windows with any integration which makes use of the A

====

[[known-issue-issue-41355]]
.The memory usage of {beats} based integrations is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage.
[%collapsible]
====

*Details*

In 8.15, events in the memory queue are not freed when they are acknowledged (as intended), but only when they are overwritten by later events in the queue buffer. This means for example if a configuration has a queue size of 5000, but the input data is low-volume and only 100 events are active at once, then the queue will gradually store more events until reaching 5000 in memory at once, then start replacing those with new events.

See {beats} issue link:https://github.com/elastic/beats/issues/40705[#40705].

*Impact* +

Memory usage may be higher than in previous releases depending on the throughput of {agent}. A fix is planned for 8.15.4.

- The worst memory increase is for low-throughput configs with large queues.
- For users whose queues were already sized proportionate to their throughput, memory use is increased but only marginally.
- Affected users can mitigate the higher memory usage by lowering their queue size.

====

[discrete]
[[new-features-8.15.0]]
=== New features
Expand Down