Example request: Expand object detection predictions to the original image

[callmephilip]

What is the task?
Object detection

Is this example for a specific model?
Not as far as I can tell. What I am wondering about is how to adapt results from the inference phase of the object detection scenario to be displayable over the original image as opposed to transformed sample. Here's is some code to illustrate

def predict_from_url(image_url):
    # this is just a helper defined previously
    img = image_from_url(image_url)

    infer_tfms = tfms.A.Adapter([*tfms.A.resize_and_pad(384), tfms.A.Normalize()])
    infer_ds = Dataset.from_images([img], infer_tfms)
    infer_dl = faster_rcnn.infer_dl(infer_ds, batch_size=1)
    
    batch, samples = faster_rcnn.build_infer_batch(infer_ds)
    preds = faster_rcnn.predict(model=model, batch=batch)

    imgs = [sample["img"] for sample in samples]
  
    # This works as expected for a sample which are transformed original image (resized and padded)
    #  I would like to understand how to transform BBox'es returned in predictions so i can overlay them over the 
    #  original image 
    show_preds(
        samples=imgs,
        preds=preds,
        class_map=data_class_map,
        denormalize_fn=denormalize_imagenet,
        show=True,
    )

---

Don't remove
Main issue for examples: #39

[callmephilip]

I have tried the following so far

# grab bounding boxes returned via inference call
bboxes = [[bbox.xmin, bbox.ymin, bbox.xmax - bbox.xmin, bbox.ymax - bbox.ymin]]

# attempt to set up a reverse compose using height and width of the original image 
reverse_transform = tfms.A.Compose(
  [*tfms.A.resize_and_pad([height,width])],
  bbox_params=tfms.A.BboxParams(format='coco', label_fields=['category_ids']),
)

# try reverse transform and include bounding boxes
reversed = reverse_transform(image=sample, bboxes=bboxes, category_ids=category_ids)

Which produces the following result:

This is a sample from inference with what looks like a pretty good bbox

Here is the original image with bbox returned via reverse transform

As you can see, there is a bit of a discrepancy there. I am wondering what is the right way to reverse these transforms.

[lgvaz]

I think the easiest solution for now, before we integrate this on the API, is to do it manually. You just need to multiply each bbox coordinate with it's corresponding re-scale factor (original_img_size / transformed_img_size), just be sure to get X and Y coordinates correct and be careful with the padding.

[tumbleintoyourheart]

ponding re-scal

Hi @lgvaz, do you have any working examples of this?

[lgvaz]

@tumbleintoyourheart not right now, but I'm sure other people will have in the community. Join the discord server and ask there =)

[FraPochetti]

This currently works thanks to the end2end_detect function.