Training with fonts

[carl-krikorian]

Hello,
First of all great work on this library, I have been having good results with it. However, I'm still trying to improve my models further and was wondering if using fonts would help.
For context, I have been using the pytorch training script in references to train my models using the command line/jupyter-notebooks.
I often use my own custom dataset of images but I also have two custom fonts .ttf files that I can use for training.
My question is, where should I put these .ttf files for them to be accessed by the training script? Or maybe should I provide the path in the command line argument like "/path/Font-1.ttf,/path/Font-2.ttf"? What's weird is that I tried just putting their names alone as well as the path and it didn't really change my results so I'm not sure they were used.
Could you expand a bit more on how this training process works with fonts and how to feed them as command line arguments? I couldn't find details about it in the documentation except with the synthesize_page and datasets (which implies, it is used for augmentation).
Thank you for your hard work and time.

[felixdittrich92]

Hi @carl-krikorian 👋 , nice that you like the lib ❤️
About fonts you have to install the fonts on your OS in linux double click on .ttf File then you can add/install the font and pass to the training script like --fonts arial.ttf 👍 @frgfm do you think we should add this detail inside the references readme ? :) (@carl-krikorian feel free to open a issue or PR for this 😅 )

Note: fonts works only with Word generator if you pass the paths to your own datasets it does nothing/ there is no augmentation

[felixdittrich92]

Hi @carl-krikorian 👋 nice :) yes anything like :
title: Using custom fonts
If you want to use extra fonts ....

👍

[carl-krikorian]

I wanted to test it out before I made the pull request and I've been encountering a small issue. I followed the steps by double clicking the .ttf file and pressing Install. I later verified that the font was correctly installed by finding it on LibreOffice Writer and it was also in the ~/.local/share/fonts/ directory.

I've been trying to run this on the command line:

python3 doctr/references/recognition/train_pytorch.py        crnn_vgg16_bn \
                                                             --epochs 5  \
                                                             --vocab english \
                                                             --batch_size=16 \
                                                             --pretrained \
                                                             --font "BORING.ttf"

But I am encountering this Exception

Traceback (most recent call last):
  File "/home/carl/Desktop/Scripts/Test/ownocr/doctr/datasets/generator/base.py", line 123, in __init__
    _ = get_font(font, 10)
  File "/home/carl/Desktop/Scripts/Test/ownocr/doctr/utils/fonts.py", line 38, in get_font
    font = ImageFont.truetype(font_family, font_size)
  File "/home/carl/anaconda3/envs/doctemp/lib/python3.7/site-packages/PIL/ImageFont.py", line 959, in truetype
    return freetype(font)
  File "/home/carl/anaconda3/envs/doctemp/lib/python3.7/site-packages/PIL/ImageFont.py", line 956, in freetype
    return FreeTypeFont(font, size, index, encoding, layout_engine)
  File "/home/carl/anaconda3/envs/doctemp/lib/python3.7/site-packages/PIL/ImageFont.py", line 248, in __init__
    font, size, index, encoding, layout_engine=layout_engine
OSError: cannot open resource

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../doctr/references/recognition/train_pytorch.py", line 468, in <module>
    main(args)
  File "../doctr/references/recognition/train_pytorch.py", line 212, in main
    T.RandomApply(T.ColorInversion(), 0.9),
  File "/home/carl/Desktop/Scripts/Test/ownocr/doctr/datasets/generator/base.py", line 125, in __init__
    raise ValueError(f"unable to locate font: {font}")
ValueError: unable to locate font: BORING.ttf

I noticed PIL was being used and tried loading the fonts using code like this:

from PIL import ImageFont
font_type = ImageFont.truetype("fonts/BORING.ttf", 18)

I tried to draw with the above font on a blank image and it worked.
Then I tried entering an incorrect path and got the OSError: cannot open resource exception again which might indicate the source of my issue is maybe the path?
Is this perhaps a mismatch with the english vocab ? Or maybe I'm just missing a few arguments in the command line 😅 ? Please advise...
PS: I am using Ubuntu 22.04, I tested with my custom font and with the boring font above from this page here.

[felixdittrich92]

Hi @carl-krikorian yep was able to reproduce this !

The working way : sudo cp BORING.ttf /usr/local/share/fonts/ only install does install the font only for the user and Pillow seems only to look in /usr/local/share/fonts/

Dont forget to run: fc-cache -f -v after copy

[carl-krikorian]

Just verified and this works! Will add this to the PR :) ! Thanks again.

[felixdittrich92]

👍