Smart image classification

[deckerst]

Discussed in #60

^{Originally posted by quantified1995 August 31, 2021}
Didn't want to add this to my previous post so I figured I'd post a separate (forgive me as I'm new to GitHub and app development)

Another incredible feature I would love to see in this app would be smart image classification. I've seen quite a few apps implement this and I consider it a make or break feature. I'm not sure to what extent the difficulty would be to implement this but it would be awesome to see.

[Subbarao6338]

Image classification based on people, animals, location, scenary etc

[flexagoon]

The tflite_flutter package has an example app implementing image classification

[DodoLeDev]

I can do some mockups about how this feature could be integrated into the app! 😋

[OkyDooky]

I was even thinking of a very simple one that could use keywords and such in the filename or other parts of the metadata to suggest tags. The app I download most of my images from (that I wish to tag in Aves) adds the artist name and the title of the work in the filename automatically. So, if I download something that mentions a character, like Zelda, it could recommend the top tags I use on other images with "Zelda" in the title, lile "Link" or "The Legend of Zelda" and so on.
This might be an easy stepping stone towards that, perhaps.

[Inky-developer]

Maybe this could be useful: https://github.com/slavabarkov/tidy

[deckerst]

Tidy seems cool but it's huge, likely because of the model. Having the model published as part of Aves is out, but it could be installed as a separate APK.

In any case, thanks for pointing it out!

[Banana-Bond]

This feature would be really helpful. Sadly I’m not a coder, so I can’t offer much technical input, but I wanted to share some examples of similar apps that I’ve used. Maybe they can inspire you. (Unfortunately all of them are closed source 😕)

1. Chat Gallery - This one is the fastest of all the on device AI based galleries I've tried. The free version is limited to classification. The paid tier allows one to perform img2img search like TIDY. This could be used for face clustering.
2. Dalle Gallery - From the app interface I can tell that this definitely uses the FOSS SimpleGallery as the base gallery interface. So SimpleGallery with smart tags. Lags and is buggy.
3. Mylio Photos- This basically is a private photo sync solution with integrated AI features. Pretty accurate classification and good face detection. However it is very cluttered, slows down the phone and generally hard to navigate.
4. Google Gallery- This is not same as Google photos. Google seems to have abandoned this (for obvious reasons- promote Google photos) but this is a surprisingly small app for what it can do. Very snappy too. It has face clustering and smart tagging (with limited features) which runs when the phone is charging.
5. Gallery - Simple and Fast - Similar to Google Gallery. Doesn't work on modern devices due to lack of updates. Side loading is the only option for newer devices it seems.

Btw, thanks a lot for putting all the love and effort into the app. It is my go-to app for viewing photos. I can't go back to my older gallery app for anything other than editing. My dad loves it too. He says the colours in the UI are interesting.

Edit: Added two more to the list.

[devSJR]

@deckerst It would really be great to have this in Aves. TIDY works pretty well, so I guess the underlying technology is reliable. But it seems that nobody is working any longer on TIDY.

[Banana-Bond]

@deckerst you might want to take a look at ente.
It's an e2e Google Photos alternative. They recently added image classification and facial recognition to their apps. They say that all the ML jobs are handled by the user's own device unlike Google Photos. All of their apps are opensource and are being actively developed. Maybe this could be of some help.