Add Feature Flag to Disable Array Preallocation for Log Record Attributes in Limited Stack Environments

[lalitb]

Fixes #1920

Changes

This PR introduces a feature flag, no-preallocate-attributes-array, that disables the default behavior of preallocating a fixed-size array for storing log record attributes. Instead of using a preallocated array, this feature flag enables the use of the vector part of GrowableArray, which allocates memory dynamically when the first entry is added.

Rationale: Kernel mode and other environments with limited stack space require careful management of stack allocations. By using a dynamically growing vector instead of preallocating a fixed-size array, we can reduce the stack footprint.

Note - We could have directly used the Vector instead of GrowableArray for this scenario, however didn't find any visible difference in stress tests with either approach, so kept GrowableArray to be consistent.

Merge requirement checklist

• CONTRIBUTING guidelines followed
• Unit tests added/updated (if applicable)
• Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
• Changes in public API reviewed (if applicable)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.9%. Comparing base (322c985) to head (8ef994b).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #2055     +/-   ##
=======================================
+ Coverage   77.3%   77.9%   +0.5%     
=======================================
  Files        124     121      -3     
  Lines      21282   21136    -146     
=======================================
- Hits       16472   16471      -1     
+ Misses      4810    4665    -145     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[cijothomas]

Alternate suggestion:
disable_stack_log_attributes

"Preallocate" is potentially misleading, we can pre-allocate in heap as well.

[cijothomas]

please also add a changelog entry for this. There is a general lacking of documentaion for each of the feature flags, but that is something we need to work on independent of this PR.

[utpilla]

Rationale: Kernel mode and other environments with limited stack space require careful management of stack allocations. By using a dynamically growing vector instead of preallocating a fixed-size array, we can reduce the stack footprint.

Do we have users who have tried to run OpenTelemetry Rust in Kernel mode? I'm asking mainly to understand if the lack of this feature is an actual blocker for anyone.

[lalitb]

Do we have users who have tried to run OpenTelemetry Rust in Kernel mode?

We don't yet support kernel mode, because of usage of std constructs. That needs to be addressed separately. Irrespective of that, kernel stack size is limited (8kb in x32 and 16 kb in x64) for Linux kernel, so this feature-flag would become relevant once the kernel support is there.

[cijothomas]

Do we have users who have tried to run OpenTelemetry Rust in Kernel mode?

We don't yet support kernel mode, because of usage of std constructs. That needs to be addressed separately. Irrespective of that, kernel stack size is limited (8kb in x32 and 16 kb in x64) for Linux kernel, so this feature-flag would become relevant once the kernel support is there.

Is it better to delay adding this feature flag until we reach that state? Just to keep overall feature-flags under control.
We can have a tracking issue to gauge the overall interest in the feature (there could be other scenarios beyond kernel mode code)

[utpilla]

Is it better to delay adding this feature flag until we reach that state?

I would be in favor of that.

Irrespective of that, kernel stack size is limited (8kb in x32 and 16 kb in x64) for Linux kernel, so this feature-flag would become relevant once the kernel support is there.

It'd be good to try it out first and check if we run into issues. (once we start supporting this scenario).

We allocate 10 instances of Option<(Key, AnyValue)> on stack. The size of Option<(Key, AnyValue)> should be around 48 bytes. 480 bytes of stack allocation for a stack size of 8kb/16kb is probably okay?

[lalitb]

Is it better to delay adding this feature flag until we reach that state?

sure, I actually prefer that. The feature flag requirement was added in #1920, so created the PR :)

We allocate 10 instances of Option<(Key, AnyValue)> on stack. The size of Option<(Key, AnyValue)> should be around 48 bytes. 480 bytes of stack allocation for a stack size of 8kb/16kb is probably okay?

I think tracing needs to be conservative for any stack allocation in kernel mode, specifically these pre-allocations in stack. Thinking about the deep call chains, and recursion in actual kernel mode application, the stack size can grow fast. But, there would be other places we need to look too, and good to revisit once we think about kernel-mode :)

[lalitb]

closing as discussed above.