Multi-Modal Retrieval using Cohere Multi-Modal Embeddings

[WASasquatch]

This is a great article/tutorial, the idea for this was great. However, I find it very strange that we're teaching people to just arbitrarily delete images, and thus, arbitrary data, because of the 5mb image limit. It would be far more appropriate to show down-scaling images within max size and compression that would inherently lead to images under 5mb. That way no data is randomly loss because of removed-from-task script functionality. Data could be lost without it immediately being obvious without having previously manually downloaded wiki images and inspected them.

import os
from PIL import Image

def process_images(directory, max_size=(1280, 768), quality=85):
    """
    Discovers images above 5MB in the specified directory, down-scales them to a max size,
    compresses them as JPEG, and deletes the original file, replacing it with the processed image.
    
    :param directory: Path to the directory to search for images.
    :param max_size: Tuple indicating max width/height for down-scaling.
    :param quality: Quality of the JPEG compression (1-100).
    """
    for root, _, files in os.walk(directory):
        for file in files:
            file_path = os.path.join(root, file)
            try:
                if os.path.getsize(file_path) > 5 * 1024 * 1024:
                    with Image.open(file_path) as img:
                        img = img.convert("RGB")
                        img.thumbnail(max_size)
                        new_file_path = os.path.splitext(file_path)[0] + ".jpg"
                        img.save(new_file_path, "JPEG", quality=quality)
                        if os.path.exists(file_path):
                            os.remove(file_path)
                        
                        print(f"Processed and replaced: {file_path} -> {new_file_path}")
            except Exception as e:
                print(f"Failed to process {file_path}: {e}")


process_images("/path/to/images")