Can we hide the private information in image or document?

[khawar-islam]

Hello,

During OCR process, some images contain a wide range of private information that may be shared unintentionally such as
e.g. email-address, picture-id or finger-print. Would it be possible to hide these type of information?

For example: phone no, Home Address, Birth Date, Passport and etc

[felixdittrich92]

Hi @khawar-islam 👋,

you mean to exclude from the OCR process ?
This would be more a post processing step for example you could try something with regex or train a Named Entity Recognition (NER) to classify the tokens you want to exclude afterwards.
If all of your documents follow the same pattern you could also try to exclude some boxes by there position

[khawar-islam]

@felixdittrich92 thank you for your reply.

I dont want to exclude like in doctr, we have face detection function so after face detection we can easily hide the face by making square box on face. In same way, we have passport number or many things we need to hide during OCR process.

Can you please give me the example for hiding passport number in document? So doctr check all the documents and hide the passport number with dark rectangle colour.

Code

for img in glob.glob(testing_path + '*.*'):
    if img.endswith(".pdf"):
        doc = DocumentFile.from_pdf(img)
    else:
        doc = DocumentFile.from_images(img)

    img_bgr_rgb = cv2.imread(img)
    file_Name = os.path.basename(img)

    image = DocumentFile.from_images(img)

    model = ocr_predictor('db_mobilenet_v3_large', 'crnn_mobilenet_v3_large', pretrained=True)

    result = model(image)

    export = result.export()

    fig = visualize_page(result.pages[0].export(), image[0], interactive=True)
    plt.savefig(os.path.join(result_path, file_Name)) 

[felixdittrich92]

You compare two different things (object detection (faces) and on the other hand text detection and recognition) for faces it is clear, but how do you think should OCR know which is for example the passport number ? 😅
As explained in my last answer there are several ways for example (not directly into docTR you have to do some steps on your own :) ):

1. after OCR classify the extracted word values (text) then grab the corresponding box values and mask the position
2. If your documents have a fixed layout (follows a pattern like passports) you could use the box values for masking otherwise you can train a LayoutLanguage model https://huggingface.co/docs/transformers/model_doc/layoutlmv3 and use this as post-processing step to classify the boxes, and it's corresponding text value. (you can easily create a dataset for the LayoutLM with docTR)

[khawar-islam]

@felixdittrich92 thank you again for the clarification. I am thinking that there will be a single model which can detect all types of passport numbers. I understand your idea and I will implement your idea. thank you @felixdittrich92