Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Post processors to give up on attempts + check on GS processing time on OCR #86

Open
DiegoPino opened this issue Mar 26, 2024 · 2 comments
Assignees
Labels
enhancement New feature or request help wanted Extra attention is needed Logging Keeping track of what we do (or not do) Post processor Plugins The ones with a ->run() method queue FIFO
Milestone

Comments

@DiegoPino
Copy link
Member

DiegoPino commented Mar 26, 2024

What?

A few things.

  • First I'm not sure if the "timeout" we have set per Processor is being respected. I just saw an OCR being stuck for a day. Code might have changed in PHP 8.1 that requires me to double check if we are enforcing this. I have a test file I can use
  • Also, a timed out Processor might re-enqueue itself. Chances are if it failed once, twice, 3 times might not solve the issue. Adding a checkbox that logs the attempt and gives up would be idea
  • On Hydroponics. I should be able also to report what is running/how long it has been running and allow an admin to intervene and kill the process (not hydroponics but kill the child process).
  • I should also be able to mark further processing of the same ADO (e.g pages of a PDF) fruitless
  • Also GS. I have seen on complex (not linearized) PDFs GS failing to export an image from a PDF and ending in a day long loop. Since we have a way of killing OCR (which I always thought was the issue with complex stuff) we should actually look more closely to PDFalto and GS ...and do the same. + Log it. Like LOG it hard

@alliomeria this is re: today's queue issues

@DiegoPino DiegoPino self-assigned this Mar 26, 2024
@DiegoPino DiegoPino added enhancement New feature or request help wanted Extra attention is needed queue FIFO Logging Keeping track of what we do (or not do) Post processor Plugins The ones with a ->run() method labels Mar 26, 2024
@DiegoPino DiegoPino added this to the 0.8.0 milestone Mar 26, 2024
@DiegoPino DiegoPino changed the title Allow Post processors to give up on attempts (Might be have to be moved to Strawberryfield bc of Hydroponics) Allow Post processors to give up on attempts + check on GS processing time on OCR Mar 29, 2024
@alliomeria
Copy link
Contributor

Giving this a bump @DiegoPino, we saw a related happening today related to a color conversion issue jamming up a queue.

@DiegoPino
Copy link
Member Author

@alliomeria PHP 8.1 is not handling timeouts as it used to in the same way, which means a stuck process is not timing out. I will have to modify the methods we use. Killing is still working. But also our swapping between queues when something fails is leading to CRON taking over long running processes that should only be handled (e.g OCR) by the background. So will open another issue to follow up with this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed Logging Keeping track of what we do (or not do) Post processor Plugins The ones with a ->run() method queue FIFO
Projects
None yet
Development

No branches or pull requests

2 participants