Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update lightgrep scanner for bulk_extractor 2.0 #421

Draft
wants to merge 31 commits into
base: main
Choose a base branch
from
Draft

Update lightgrep scanner for bulk_extractor 2.0 #421

wants to merge 31 commits into from

Commits on May 2, 2023

  1. R - delete lightgrep scanners not ported, edit Makefile.am accordingly

    @juliapal
    juliapal committed May 2, 2023
    Configuration menu
    Copy the full SHA
    43f5303 View commit details
    Browse the repository at this point in the history

  2. F - skeleton of new lightgrep scanner for BE2.0

    @juliapal
    juliapal committed May 2, 2023
    Configuration menu
    Copy the full SHA
    287b882 View commit details
    Browse the repository at this point in the history

  3. R - comment out unused phases

    @juliapal
    juliapal committed May 2, 2023
    Configuration menu
    Copy the full SHA
    dc034c8 View commit details
    Browse the repository at this point in the history

  4. F!! - initialization of Lightgrep Controller

    @juliapal
    juliapal committed May 2, 2023
    Configuration menu
    Copy the full SHA
    d638a2a View commit details
    Browse the repository at this point in the history

  5. R - If PHASE_INIT2 happens >1, Get() causes Fsm to be empty on subseq… 

    …uent passes
    @juliapal
    juliapal committed May 2, 2023
    Configuration menu
    Copy the full SHA
    8ecacd1 View commit details
    Browse the repository at this point in the history

  6. F - simplify addUserPatterns for now

    @juliapal
    juliapal committed May 2, 2023
    Configuration menu
    Copy the full SHA
    ee7b242 View commit details
    Browse the repository at this point in the history

  7. F - make regcomp, gotHit, HitData, and scan work with new skeleton

    @juliapal
    juliapal committed May 2, 2023
    Configuration menu
    Copy the full SHA
    7e1eefa View commit details
    Browse the repository at this point in the history

  8. R - Prog may not be initialized before numPatterns is called

    @juliapal
    juliapal committed May 2, 2023
    Configuration menu
    Copy the full SHA
    c629d3a View commit details
    Browse the repository at this point in the history

  9. R - fix declarations

    @juliapal
    juliapal committed May 2, 2023
    Configuration menu
    Copy the full SHA
    72c69a8 View commit details
    Browse the repository at this point in the history

  10. F - Append lightgrep feature recorder to feature defs

    @juliapal
    juliapal committed May 2, 2023
    Configuration menu
    Copy the full SHA
    99bc030 View commit details
    Browse the repository at this point in the history

Commits on May 4, 2023

  1. Delete unused scanner

    @juliapal
    juliapal committed May 4, 2023
    Configuration menu
    Copy the full SHA
    1c1c4d0 View commit details
    Browse the repository at this point in the history

  2. give write_buf good args

    @juliapal
    juliapal committed May 4, 2023
    Configuration menu
    Copy the full SHA
    4579d9c View commit details
    Browse the repository at this point in the history

  3. Result is positive upon success

    @juliapal
    juliapal committed May 4, 2023
    Configuration menu
    Copy the full SHA
    7553305 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2023

  1. F!! enable mulitple patterns passed through CLI

    @juliapal
    juliapal committed May 9, 2023
    Configuration menu
    Copy the full SHA
    a3616e0 View commit details
    Browse the repository at this point in the history

  2. F - add user_files param to addUserPatterns to avoid scanner config a… 

    …ccess
    @juliapal
    juliapal committed May 9, 2023
    Configuration menu
    Copy the full SHA
    5097c1c View commit details
    Browse the repository at this point in the history

  3. F - Handle user files with parsed patterns in lightgrep scanner

    @juliapal
    juliapal committed May 9, 2023
    Configuration menu
    Copy the full SHA
    92b7f75 View commit details
    Browse the repository at this point in the history

  4. Remove unused or commented code

    @juliapal
    juliapal committed May 9, 2023
    Configuration menu
    Copy the full SHA
    c553d2b View commit details
    Browse the repository at this point in the history

Commits on May 11, 2023

  1. b - lightgrep should run even if find is disabled

    @juliapal
    juliapal committed May 11, 2023
    Configuration menu
    Copy the full SHA
    707466b View commit details
    Browse the repository at this point in the history

  2. F - enable histogram for lightgrep

    @juliapaluch
    juliapaluch committed May 11, 2023
    Configuration menu
    Copy the full SHA
    48b9ca3 View commit details
    Browse the repository at this point in the history

  3. get rid of comments & unused code

    @juliapaluch
    juliapaluch committed May 11, 2023
    Configuration menu
    Copy the full SHA
    e7ac804 View commit details
    Browse the repository at this point in the history

  4. F - delete Fsm if not deleted by regcomp

    @juliapaluch
    juliapaluch committed May 11, 2023
    Configuration menu
    Copy the full SHA
    8cdc2f1 View commit details
    Browse the repository at this point in the history

  5. b - don't declare initScan

    @juliapaluch
    juliapaluch committed May 11, 2023
    Configuration menu
    Copy the full SHA
    bb88ad8 View commit details
    Browse the repository at this point in the history

  6. F - remove Scanner "global" variable

    @juliapaluch
    juliapaluch committed May 11, 2023
    Configuration menu
    Copy the full SHA
    90bb108 View commit details
    Browse the repository at this point in the history

  7. remove unused includes

    @juliapaluch
    juliapaluch committed May 11, 2023
    Configuration menu
    Copy the full SHA
    8c143b3 View commit details
    Browse the repository at this point in the history

  8. F - throw exception when lightgrep fails to parse pattern

    @juliapaluch
    juliapaluch committed May 11, 2023
    Configuration menu
    Copy the full SHA
    f553e62 View commit details
    Browse the repository at this point in the history

  9. better error messages

    @juliapaluch
    juliapaluch committed May 11, 2023
    Configuration menu
    Copy the full SHA
    799221b View commit details
    Browse the repository at this point in the history

Commits on May 12, 2023

  1. F - avoid unnecessary repeated heap alloc/dealloc

    @juliapaluch
    juliapaluch committed May 12, 2023
    Configuration menu
    Copy the full SHA
    0ca43ec View commit details
    Browse the repository at this point in the history

Commits on May 30, 2023

  1. F!! - Reverting use of thread local because it seems to cause a perfo… 

    …rmance regression. The timings below are from the following command:

./src/bulk_extractor -F ../lightgrep/pytest/keys/shuf10.txt -Z -o ~/be_timed_output_without_thread_local_`printf %04d $i` -E scan_lightgrep ~/ev/terry-2009-12-11-002.E01

Thread_local?	Clocktime (Min.)	Clocktime (Max.)	Clocktime (Average)	Scan Lightgrep Time (Min.)	Scan Lightgrep Time (Max.)	Scan Lightgrep Time (Average)
FALSE	162.965479	168.628229	164.2545712	494.810946	528.368114	504.1799554
TRUE	163.681386	173.587754	167.233617	499.815450	532.324335	516.4901762

This reverts commit 0ca43ec.
    @juliapaluch
    juliapaluch committed May 30, 2023
    Configuration menu
    Copy the full SHA
    007b9c9 View commit details
    Browse the repository at this point in the history

  2. F - update scanner name and version

    @juliapaluch
    juliapaluch committed May 30, 2023
    Configuration menu
    Copy the full SHA
    fb0e42e View commit details
    Browse the repository at this point in the history

  3. a - Delete superfluous whitespace

    @juliapaluch
    juliapaluch committed May 30, 2023
    Configuration menu
    Copy the full SHA
    5e55041 View commit details
    Browse the repository at this point in the history

  4. a - formatting, fix inaccurate comments

    @juliapaluch
    juliapaluch committed May 30, 2023
    Configuration menu
    Copy the full SHA
    16e8eeb View commit details
    Browse the repository at this point in the history