(Feature Request) PyMuPDF for pdf parsing

[aditirao7]

Currently we are using PDFMiner for text extraction from the pdf. There are other libraries that are faster like PyMuPDF that we could consider. It'll also help generate the report faster for the web app that way.

Reference:
https://github.com/py-pdf/benchmarks#pdf-library-benchmarks

[marshalmiller]

@aditirao7 I'm sorry it took so long for me to address this. I think this is a great idea. I appreciate the benchmarks and was hoping to find a way to speed up the web app too. My delay was that I wanted to confirm that their license was compatible with ours. I'm confident that it is now. I think we should move forward with this.

[aditirao7]

No worries, I've managed to migrate to PyMuPDF and it works for the test pdfs in the repo. Do you have any other pdfs I can test on that tend to cause errors? Also I am not sure how to make changes to the unit tests but will try anyway.

[marshalmiller]

@aditirao7 There are some random PDFs in this folder. I don't think that any cause more trouble but they are a better representation of what we would get. https://mega.nz/folder/uwBxVSzS#lpBtSz49E9dqHtmrQwp0Ig

Thank you so much for this work.

[aditirao7]

@marshalmiller I was looking into the unit tests and it seems like many of them are not asserting correctly, did these tests ever successfully run?