Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Inference Accelerated PDF batch parsing #106

Open
wants to merge 85 commits into
base: main
Choose a base branch
from
Open

Inference Accelerated PDF batch parsing #106

wants to merge 85 commits into from

Commits on Jul 23, 2024

  1. sync

    @veya2ztn
    veya2ztn committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    3d5d93a View commit details
    Browse the repository at this point in the history

  2. add rough layout module

    @veya2ztn
    veya2ztn committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    8bbef33 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. add batch processing

    @veya2ztn
    veya2ztn committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    ed136d4 View commit details
    Browse the repository at this point in the history

  2. test in S1

    @veya2ztn
    veya2ztn committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    c334a66 View commit details
    Browse the repository at this point in the history

  3. alignmented

    @veya2ztn
    veya2ztn committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    10ccb40 View commit details
    Browse the repository at this point in the history

  4. reformat to match data group format

    @veya2ztn
    veya2ztn committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    d8dded3 View commit details
    Browse the repository at this point in the history

  5. sync

    @veya2ztn
    veya2ztn committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    f655e6b View commit details
    Browse the repository at this point in the history

  6. mnerge

    @veya2ztn
    veya2ztn committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    2816698 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. lets try another size assign

    @veya2ztn
    veya2ztn committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    8681980 View commit details
    Browse the repository at this point in the history

  2. sync

    @veya2ztn
    veya2ztn committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    5037ed0 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. sync

    @veya2ztn
    veya2ztn committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    426f17c View commit details
    Browse the repository at this point in the history

  2. what new?

    @veya2ztn
    veya2ztn committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    d50097a View commit details
    Browse the repository at this point in the history

  3. merge

    @veya2ztn
    veya2ztn committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    b195c46 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. add ocr batch processing

    @veya2ztn
    veya2ztn committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    b40a558 View commit details
    Browse the repository at this point in the history

  2. add pytorchocr

    @veya2ztn
    veya2ztn committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    e727dfa View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. we pass the layout+mfd+det+rec test

    @veya2ztn
    veya2ztn committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    addccda View commit details
    Browse the repository at this point in the history

  2. sync

    @veya2ztn
    veya2ztn committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    1b60a5e View commit details
    Browse the repository at this point in the history

  3. add weight

    @veya2ztn
    veya2ztn committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    a1cef0a View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. sync

    @veya2ztn
    veya2ztn committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    76cb7f1 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2024

  1. add rec batch model

    @veya2ztn
    veya2ztn committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    3f007e6 View commit details
    Browse the repository at this point in the history

  2. sync

    @veya2ztn
    veya2ztn committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    1446e75 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. lets use tensorRT

    @veya2ztn
    veya2ztn committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    49ddeba View commit details
    Browse the repository at this point in the history

  2. try to convert layoutlmV3 to onnx and tensorrt

    @veya2ztn
    veya2ztn committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    aeb23f6 View commit details
    Browse the repository at this point in the history

  3. add sample image

    @veya2ztn
    veya2ztn committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    d422430 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. our layoutLMv3 use CasadeROIalign which is not caplabe to detectron2 … 

    …to tensorRT script. lets do tensor rt only for the layoutLMv3 backbone
    @veya2ztn
    veya2ztn committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    5031624 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. sync

    @veya2ztn
    veya2ztn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    81e5a21 View commit details
    Browse the repository at this point in the history

  2. lets seperate the postprocess

    @veya2ztn
    veya2ztn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    00b04d9 View commit details
    Browse the repository at this point in the history

  3. it same use numpy backend faster

    @veya2ztn
    veya2ztn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    b22a308 View commit details
    Browse the repository at this point in the history

  4. lets load tensorrt MFD model

    @veya2ztn
    veya2ztn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    56d4ca6 View commit details
    Browse the repository at this point in the history

  5. we find the YOLO tensorRT has strict on the input size

    @veya2ztn
    veya2ztn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    d20f0b0 View commit details
    Browse the repository at this point in the history

  6. we find the YOLO tensorRT has strict on the input size

    @veya2ztn
    veya2ztn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    ac2a33c View commit details
    Browse the repository at this point in the history

  7. sync

    @veya2ztn
    veya2ztn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    1162692 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'ONNX2TensorRT' of https://github.com/veya2ztn/PDF-Extra… 

    …ct-Kit into ONNX2TensorRT
    @veya2ztn
    veya2ztn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    9f767b2 View commit details
    Browse the repository at this point in the history

  9. pad batch to avoid tensorRT mdf wrong

    @veya2ztn
    veya2ztn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    ff3cccf View commit details
    Browse the repository at this point in the history

  10. pass tensorRT and compiled model

    @veya2ztn
    veya2ztn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    f989a21 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. sync

    @veya2ztn
    veya2ztn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    a546387 View commit details
    Browse the repository at this point in the history

  2. check

    @veya2ztn
    veya2ztn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    cd99977 View commit details
    Browse the repository at this point in the history

  3. merge

    @veya2ztn
    veya2ztn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    827ca37 View commit details
    Browse the repository at this point in the history

  4. add async implement

    @veya2ztn
    veya2ztn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    c11d603 View commit details
    Browse the repository at this point in the history

  5. add smart batch size judge

    @veya2ztn
    veya2ztn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    ad3733c View commit details
    Browse the repository at this point in the history

  6. pass async

    @veya2ztn
    veya2ztn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    d195745 View commit details
    Browse the repository at this point in the history

  7. pass async

    @veya2ztn
    veya2ztn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    b3f9334 View commit details
    Browse the repository at this point in the history

  8. make det through inner batch size to avoid too large tensor

    @veya2ztn
    veya2ztn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    9573dfd View commit details
    Browse the repository at this point in the history

  9. sync

    @veya2ztn
    veya2ztn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    7fdd259 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. before code may make the dataiter skip at final page.

    @veya2ztn
    veya2ztn committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    afad9bd View commit details
    Browse the repository at this point in the history

  2. we now made aync mode online

    @veya2ztn
    veya2ztn committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    159cc1a View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. add batch run script

    @veya2ztn
    veya2ztn committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    887f31c View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. lets do an async data loader into memory

    @veya2ztn
    veya2ztn committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    6f96a89 View commit details
    Browse the repository at this point in the history

  2. stepped rough rec. Now the bottleneck appear dataloader setup

    @veya2ztn
    veya2ztn committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    0faba59 View commit details
    Browse the repository at this point in the history

  3. stepped rough rec. Now the bottleneck appear dataloader setup

    @veya2ztn
    veya2ztn committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    2d7405f View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. Not iterableDataset

    @veya2ztn
    veya2ztn committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    8127be2 View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2024

  1. sync

    @veya2ztn
    veya2ztn committed Aug 18, 2024
    Configuration menu
    Copy the full SHA
    241fc4a View commit details
    Browse the repository at this point in the history

  2. we add no paddle depend workflow and pass on beijing server

    @veya2ztn
    veya2ztn committed Aug 18, 2024
    Configuration menu
    Copy the full SHA
    cf2e549 View commit details
    Browse the repository at this point in the history

  3. some script is server based and be removed

    @veya2ztn
    veya2ztn committed Aug 18, 2024
    Configuration menu
    Copy the full SHA
    1b30bf3 View commit details
    Browse the repository at this point in the history

  4. add checklog

    @veya2ztn
    veya2ztn committed Aug 18, 2024
    Configuration menu
    Copy the full SHA
    b0a1d93 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. forget add no_paddle object

    @veya2ztn
    veya2ztn committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    f8fc7f1 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2024

  1. lets use reverse ssh port transfer the lock

    @veya2ztn
    veya2ztn committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    1a5d8bc View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. we pass the faster rec inference framework

    @veya2ztn
    veya2ztn committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    f1efc04 View commit details
    Browse the repository at this point in the history

  2. merge

    @veya2ztn
    veya2ztn committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    239a5fd View commit details
    Browse the repository at this point in the history

  3. fix a bug for image crop, for early version data like layoutV1, pleas… 

    …e mind the height/weight is different the latest version
    @veya2ztn
    veya2ztn committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    9194031 View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2024

  1. clean and reconstruct

    @veya2ztn
    veya2ztn committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    ee54b1c View commit details
    Browse the repository at this point in the history

  2. pass batchrun

    @veya2ztn
    veya2ztn committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    bce8204 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. merge

    @veya2ztn
    veya2ztn committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    6a0968e View commit details
    Browse the repository at this point in the history

  2. add README.md

    @veya2ztn
    veya2ztn committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    0759cc3 View commit details
    Browse the repository at this point in the history

  3. update README

    @veya2ztn
    veya2ztn committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    2e78c93 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. add batch det

    @veya2ztn
    veya2ztn committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    afabaff View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. add missing part fixing

    @veya2ztn
    veya2ztn committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    bb18105 View commit details
    Browse the repository at this point in the history

  2. sync

    @veya2ztn
    veya2ztn committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    b734ec8 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'ONNX2TensorRT' of https://github.com/veya2ztn/PDF-Extra… 

    …ct-Kit into ONNX2TensorRT
    @veya2ztn
    veya2ztn committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    3221eb0 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. update rec, we forget consider exclude the mfd part

    @veya2ztn
    veya2ztn committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    fc5adc0 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'ONNX2TensorRT' of https://github.com/veya2ztn/PDF-Extra… 

    …ct-Kit into ONNX2TensorRT
    @veya2ztn
    veya2ztn committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    d3f5d4a View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. add replace mode and full data save mode

    @veya2ztn
    veya2ztn committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    35d28a9 View commit details
    Browse the repository at this point in the history

  2. merge

    @veya2ztn
    veya2ztn committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    ec9b1be View commit details
    Browse the repository at this point in the history

  3. Merge branch 'ONNX2TensorRT' of https://github.com/veya2ztn/PDF-Extra… 

    …ct-Kit into ONNX2TensorRT
    @veya2ztn
    veya2ztn committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    e6cdab3 View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. sync

    @veya2ztn
    veya2ztn committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    1817e53 View commit details
    Browse the repository at this point in the history

  2. sync

    @veya2ztn
    veya2ztn committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    73bae56 View commit details
    Browse the repository at this point in the history

  3. sync

    @veya2ztn
    veya2ztn committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    7f510b0 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. add processing

    @veya2ztn
    veya2ztn committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    9a89ff5 View commit details
    Browse the repository at this point in the history

  2. Update README.md

    @veya2ztn
    veya2ztn committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    9ca6096 View commit details
    Browse the repository at this point in the history

  3. add some script

    @veya2ztn
    veya2ztn committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    38df666 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. Update README.md

    @veya2ztn
    veya2ztn committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    6e739c1 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. update for local lock

    @veya2ztn
    veya2ztn committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    4307e18 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. add Fail Case

    @veya2ztn
    veya2ztn committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    52b1a8e View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2024

  1. sync

    @veya2ztn
    veya2ztn committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    59cdf72 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2024

  1. add test

    @veya2ztn
    veya2ztn committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    36b4104 View commit details
    Browse the repository at this point in the history