Image orientation not correctly detected

[ep0p]

Hi,

I'm working on an OCR pipeline where I need to detect image orientation before applying layout tasks. I'm using Doctr for this, but I'm getting inconsistent results with some vertical images. For example, the attached image is returning an orientation angle of 180°.

Any insights on what might be going wrong?

[felixdittrich92]

Hi @ep0p 👋,

We also saw such cases, the only and easiest way here is to fine tune the model .. luckily everything you need are plain images (without rotations) because the labels are augmented so that's really straight forward :)

[ep0p]

scrap that, I think found it: page_orientation_predictor() :D

[felixdittrich92]

:D

Yeah currently we support only:
mobilenet_v3_small_page_orientation
and
mobilenet_v3_small_crop_orientation
as architecture for this task

Additional you can modify the auto augmentation part to your needs, currently we additional augment in the range (-180, 180) so full rotation

doctr/references/classification/train_pytorch_orientation.py

Line 44 in 225210c

def rnd_rotate(img: torch.Tensor, target):

if you only have to deal with exact [-90, 0, 90, 180] cases you could comment the following part:

# augment the angle randomly with a probability of 0.5
    if np.random.rand() < 0.5:
        angle += float(np.random.choice(np.arange(-25, 25, 5)))

[ep0p]

hmmm i just finished 3 fine-tuning attempts, with a ds of 300images/train and 100images/val but the results are not that good; i must be doing something wrong

python references/classification/train_pytorch_orientation.py mobilenet_v3_small_page_orientation --type page --train_path /home/epop/DATASET/GOT/output_dataset_20k/train --val_path /home/epop/DATASET/GOT/output_dataset_20k/val --name mobilenet_v3_small_page_orientation_custom_v3 --epochs 20 --b 256 --early-stop

the learning rate is decreasing properly, but the accuracy is fluctuating and never depassing 35%
i have tried 20, 30 , 50 epochs

[felixdittrich92]

@ep0p --pretrained missing :D

[ep0p]

i have confirmed, again, the problem is in chair not in computer
thanks a lot!