detect/multi-tenant: Eliminate crashes processing tenant config

[jlucovsky]

Continuation of #9056

This PR eliminates crashes during parallel loading of tenant configurations by serializing processing of the .config files

• reference.config
• threshold.config
• classification.config

Link to redmine ticket: 6047

Describe changes:

• Formatting fixups
• Removal of commit that triggered "uninitialized variable" errors.

Provide values to any of the below to override the defaults.

To use a pull request use a branch name like pr/N where N is the pull request number.

SV_REPO=
SV_BRANCH=
SU_REPO=
SU_BRANCH=
LIBHTP_REPO=
LIBHTP_BRANCH=

[codecov]

Codecov Report

Merging #9060 (b8c5728) into master (643e674) will increase coverage by 0.01%.
The diff coverage is 75.86%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9060      +/-   ##
==========================================
+ Coverage   82.32%   82.34%   +0.01%     
==========================================
  Files         969      969              
  Lines      273655   273662       +7     
==========================================
+ Hits       225292   225337      +45     
+ Misses      48363    48325      -38     

Flag	Coverage Δ
fuzzcorpus	64.57% <48.27%> (+0.04%)	⬆️
suricata-verify	60.61% <72.41%> (-0.03%)	⬇️
unittests	62.90% <62.06%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

[catenacyber]

S-V : find . -name "*.pcap" -perm +111

[suricata-qa]

Information: QA ran without warnings.

Pipeline 14764

[victorjulien]

there are far more call sites for pcre2_substring_get_bynumber, including some at packet time (pcre keyword), so if pcre2_substring_get_bynumber truly isn't thread safe, we need a much bigger patch.

I'm missing a good explanation of the thread safety issue, what is the issue?

[victorjulien]

see comment

[jlucovsky]

The static regex structures within the reference, threshold, and classification file processing (static pcred2_code *regex and static pcred2_match_data for example) were being used concurrently.

This PR prevents concurrent access -- access was most often through pcre2_substring_copy_bynumber -- by the multiple loader threads used in MT.

[victorjulien]

The static regex structures within the reference, threshold, and classification file processing (static pcred2_code *regex and static pcred2_match_data for example) were being used concurrently.

This PR prevents concurrent access -- access was most often through pcre2_substring_copy_bynumber -- by the multiple loader threads used in MT.

This type of explanation belongs in the commit.

[victorjulien]

The static regex structures within the reference, threshold, and classification file processing (static pcred2_code *regex and static pcred2_match_data for example) were being used concurrently.

This PR prevents concurrent access -- access was most often through pcre2_substring_copy_bynumber -- by the multiple loader threads used in MT.

If this is the case I would suggest a different solution: move these pcre2 data structures into DetectEngineCtx. Of these we have an instance per tenant.

[catenacyber]

As Victor said

If this is the case I would suggest a different solution: move these pcre2 data structures into DetectEngineCtx. Of these we have an instance per tenant.

By the way, I like the commits to use SCCalloc and make tenant_id always a uint32 :-)

[catenacyber]

I think this was solved with #9243