new(libsinsp): add len() filter transformer #2131

LucaGuerra · 2024-10-23T12:31:27Z

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area libsinsp

Does this PR require a change in the driver versions?

No

What this PR does / why we need it:

This introduces a new len() transformer which acts like you'd expect:

len(some_list) evaluates to the number of elements in the list
len(some_string) evaluates to the length of the string (excluding null terminator of course)
len(some_buffer) evaluates to the number of bytes in the buffer

This is also a handy way to check if lists are empty, since we couldn't do it before, which was apparent after a discussion with @leogr and @jasondellaluce .

Which issue(s) this PR fixes:

Fixes #2127

Special notes for your reviewer:

The implementation slightly changes the transformer interface which now takes the element flags as well
Do you see any potential corner cases with plugins?

Does this PR introduce a user-facing change?:

new(libsinsp): add len() filter transformer

Signed-off-by: Luca Guerra <[email protected]>

poiana · 2024-10-23T12:31:31Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: LucaGuerra

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [LucaGuerra]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

github-actions · 2024-10-23T12:37:24Z

Perf diff from master - unit tests

    10.68%     -1.27%  [.] sinsp_parser::reset
     4.81%     -1.22%  [.] sinsp_parser::process_event
     2.66%     +0.89%  [.] sinsp_thread_manager::find_thread
     3.40%     -0.70%  [.] sinsp_thread_manager::get_thread_ref
     0.63%     +0.68%  [.] sinsp_evt::get_ts
     2.88%     +0.63%  [.] gzfile_read
     7.16%     -0.57%  [.] sinsp::next
     1.77%     +0.57%  [.] std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
     0.84%     +0.53%  [.] std::_Hashtable<long, std::pair<long const, std::shared_ptr<sinsp_threadinfo> >, std::allocator<std::pair<long const, std::shared_ptr<sinsp_threadinfo> > >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::find
     1.41%     -0.49%  [.] sinsp_threadinfo::~sinsp_threadinfo

Heap diff from master - unit tests

peak heap memory consumption: -861B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Heap diff from master - scap file

peak heap memory consumption: -32B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Benchmarks diff from master

Comparing gbench_data.json to /root/actions-runner/_work/libs/libs/build/gbench_data.json
Benchmark                                                         Time             CPU      Time Old      Time New       CPU Old       CPU New
----------------------------------------------------------------------------------------------------------------------------------------------
BM_sinsp_split_mean                                            +0.0424         +0.0425           149           156           149           156
BM_sinsp_split_median                                          +0.0509         +0.0510           149           156           149           156
BM_sinsp_split_stddev                                          -0.4585         -0.4582             2             1             2             1
BM_sinsp_split_cv                                              -0.4805         -0.4803             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_mean                  +0.0474         +0.0475            59            62            59            62
BM_sinsp_concatenate_paths_relative_path_median                +0.0450         +0.0451            59            62            59            62
BM_sinsp_concatenate_paths_relative_path_stddev                +1.3428         +1.3438             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_cv                    +1.2367         +1.2375             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_mean                     -0.0026         -0.0025            24            24            24            24
BM_sinsp_concatenate_paths_empty_path_median                   -0.0025         -0.0024            24            24            24            24
BM_sinsp_concatenate_paths_empty_path_stddev                   +2.7429         +2.7711             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_cv                       +2.7526         +2.7805             0             0             0             0
BM_sinsp_concatenate_paths_absolute_path_mean                  +0.1000         +0.1001            57            62            57            62
BM_sinsp_concatenate_paths_absolute_path_median                +0.0933         +0.0934            56            62            56            62
BM_sinsp_concatenate_paths_absolute_path_stddev                +1.5163         +1.5176             0             1             0             1
BM_sinsp_concatenate_paths_absolute_path_cv                    +1.2876         +1.2886             0             0             0             0
BM_sinsp_split_container_image_mean                            -0.0177         -0.0176           396           389           396           389
BM_sinsp_split_container_image_median                          -0.0154         -0.0153           396           390           396           390
BM_sinsp_split_container_image_stddev                          +0.1218         +0.1215             3             4             3             4
BM_sinsp_split_container_image_cv                              +0.1420         +0.1416             0             0             0             0

codecov · 2024-10-23T12:42:40Z

Codecov Report

Attention: Patch coverage is 85.93750% with 27 lines in your changes missing coverage. Please review.

Project coverage is 74.49%. Comparing base (6322a02) to head (e26cf46).
Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
userspace/libsinsp/test/filter_transformer.ut.cpp	86.15%	18 Missing ⚠️
userspace/libsinsp/sinsp_filter_transformer.cpp	84.44%	7 Missing ⚠️
userspace/libsinsp/sinsp_filter_transformer.h	71.42%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2131      +/-   ##
==========================================
+ Coverage   74.46%   74.49%   +0.02%     
==========================================
  Files         254      254              
  Lines       33333    33472     +139     
  Branches     5707     5733      +26     
==========================================
+ Hits        24821    24934     +113     
- Misses       8481     8526      +45     
+ Partials       31       12      -19

Flag	Coverage Δ
libsinsp	`74.49% <85.93%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

userspace/libsinsp/sinsp_filter_transformer.cpp

jasondellaluce · 2024-10-23T13:06:49Z

userspace/libsinsp/sinsp_filter_transformer.cpp

 	switch(m_type) {
 	case FTR_TOUPPER: {
 		switch(t) {
 		case PT_CHARBUF:
 		case PT_FSPATH:
 		case PT_FSRELPATH:
 			// for TOUPPER, the transformed type is the same as the input type
-			return true;
+			return !is_list;


Is this potentially a breaking change? It might be that we supported tolower (and similars) with list-type fields without noticing. Maybe I'm wrong, just checking in.

That makes sense, although I never tried it. I would suggest to test and document this behavior if we want it.

Ok, I have reverted this change and I'll add a couple of tests.

jasondellaluce · 2024-10-23T13:10:30Z

userspace/libsinsp/sinsp_filter_transformer.cpp

+			return false;
+		}
+
+		for(std::size_t i = 0; i < vec.size(); i++) {


Question -- is there really a scenario where the field is not a list-type and vec.size() > 1? Should we put an assert?

I honestly do not know, but tests were using it like this.

If you can't think of a corner case (such as plugins) I'll remove the loop as it simplifies things.

Co-authored-by: Jason Dellaluce <[email protected]> Signed-off-by: Luca Guerra <[email protected]>

FedeDP · 2024-10-23T15:09:53Z

/milestone 0.19.0

LucaGuerra added 2 commits October 23, 2024 12:24

new(libsinsp): add len() filter transformer

4bf1050

Signed-off-by: Luca Guerra <[email protected]>

new(libsinsp/tests): add tests for len() filter transformer

0d0b0ce

Signed-off-by: Luca Guerra <[email protected]>

poiana added release-note kind/feature New feature or request dco-signoff: yes labels Oct 23, 2024

poiana added area/libsinsp approved labels Oct 23, 2024

poiana requested review from hbrueckner and jasondellaluce October 23, 2024 12:31

poiana added the size/XL label Oct 23, 2024

jasondellaluce reviewed Oct 23, 2024

View reviewed changes

Update userspace/libsinsp/sinsp_filter_transformer.cpp

e26cf46

Co-authored-by: Jason Dellaluce <[email protected]> Signed-off-by: Luca Guerra <[email protected]>

poiana added this to the 0.19.0 milestone Oct 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new(libsinsp): add len() filter transformer #2131

new(libsinsp): add len() filter transformer #2131

LucaGuerra commented Oct 23, 2024 •

edited

Loading

poiana commented Oct 23, 2024

github-actions bot commented Oct 23, 2024 •

edited

Loading

codecov bot commented Oct 23, 2024 •

edited

Loading

jasondellaluce Oct 23, 2024

LucaGuerra Oct 23, 2024

LucaGuerra Oct 23, 2024

jasondellaluce Oct 23, 2024

LucaGuerra Oct 23, 2024

LucaGuerra Oct 23, 2024

FedeDP commented Oct 23, 2024

new(libsinsp): add len() filter transformer #2131

Are you sure you want to change the base?

new(libsinsp): add len() filter transformer #2131

Conversation

LucaGuerra commented Oct 23, 2024 • edited Loading

poiana commented Oct 23, 2024

github-actions bot commented Oct 23, 2024 • edited Loading

Perf diff from master - unit tests

Heap diff from master - unit tests

Heap diff from master - scap file

Benchmarks diff from master

codecov bot commented Oct 23, 2024 • edited Loading

Codecov Report

jasondellaluce Oct 23, 2024

Choose a reason for hiding this comment

LucaGuerra Oct 23, 2024

Choose a reason for hiding this comment

LucaGuerra Oct 23, 2024

Choose a reason for hiding this comment

jasondellaluce Oct 23, 2024

Choose a reason for hiding this comment

LucaGuerra Oct 23, 2024

Choose a reason for hiding this comment

LucaGuerra Oct 23, 2024

Choose a reason for hiding this comment

FedeDP commented Oct 23, 2024

LucaGuerra commented Oct 23, 2024 •

edited

Loading

github-actions bot commented Oct 23, 2024 •

edited

Loading

codecov bot commented Oct 23, 2024 •

edited

Loading