[Feature Request] Image captioning to help with the evaluation process

[augusto-scarvalho]

First of all things, this repo is amazing! Thank you guys for all the hard work!

About the requests. I think that the automated merging process could benefit from image captioning as a way of monitoring the coherence between the prompt and the sampled images and also checking for artifacts/unwanted features being added. This kind of feature could also be useful if the objective is merging a model that capable of reproducing or avoiding specific concepts.

For instance, using BLIP/GIT/OFA/WD1.4 we could check if the sampled image has all the tokens prompted, measure the confidence of each recognized token and also keep track of a list of desired or undesired tokens, finishing each batch with a score of how the merge performed and use it paired with the aesthetic scorer.