assume_straight_pages vs. straighten_pages

[Adversarian]

The ocr_predictor class can take two kwargs called assume_straight_pages and straighten_pages. My understanding of the documentation is that straighten_pages=True should entail assume_straight_pages=False. However I wanted to ask to make sure when I need to use one or the other, or should I be using both when dealing with tilted/crooked documents (i.e. assume_straight_pages=False, straighten_pages=True) .

[felixdittrich92]

Hi @Adversarian 👋🏼,

straighten_pages=True should be used with assume_straight_pages=False in combination.
The specific of straighten_pages=True is, that we try to estimate the orientation angle from the document, rotate it counter clockwise to "straighten" the image and atm. process it again with the straightened image.
Without this so only assume_straight_pages=False we don't try to straighten it and process it as is (output geometry are also polygons).

Does this help ? :)

[Adversarian]

Thanks for your response. However I still don't understand what straighten_pages=False, assume_straight_pages=False does. If we're not assuming that the pages are straight, yet we're also not straightening the pages before re-running the pipeline on the image then what is it exactly that assume_straight_pages=False does?

[felixdittrich92]

In short:

with assume_straight_pages=False under the hood it's handled as quad polygons [[x1, y1], [x2, y2], [x3, y3], [x4, y4]] instead of boxes [xmin, ymin, xmax, ymax] .. this starts by extracting the coords (detection postprocessor) from the detection model results .. and goes trough the whole pipeline

So the pipeline is the same but with assume_straight_pages=False we have also to take care of rotations and compute "on polygons" instead of boxes

• additional to support this we plug in (now on main branch) 2 tiny models for general crop and page orientation classification

Try a rotated document with assume_straight_pages=False and assume_straight_pages=True to get a visual feedback.

A bit clearer ? :)

[Adversarian]

That clears everything. Thank you.

[milosacimovic]

What if the pages are almost straight, but we would benefit from polygons. In this case I am getting some weird results when rectifying crops i.e. the images that are being fed to the recognition model are flip upside down. Could that be a problem with crop orientation estimation?

Like in this issue here:
#1455

[felixdittrich92]

@milosacimovic Yeah then it indicates that the orientation model/s fail.
Added a way to load custom trained orientation models in: #1708 + docu with link to the training scripts

[milosacimovic]

Wouldn't be easier for the pipeline to have support working with almost straight pages? We still want to use polygons are crop according to polygon coordinates but there is no 90, 180 or 270 degree of rotation.
i.e. not doing the cumbersome job of estimating the orientation, unnecessary overhead.

Because in almost straight pages case all we need to do in extract_rcrops is to just map correctly the source and destination points and we have a proper crop to feed to the recognition pipeline.