OK, maybe not everything, but it is two things in one: a tool for manually annotating images and a tool for training and running deep detectors. Those two things are coupled in an active learning loop: the human annotates a few images, the system trains a model, that model is used to make predictions and to select more images for the human to annotate, etc.
More generally, AIde is a modular Web framework for labeling image datasets with AI assistance. AIde is configurable for a variety of tasks, but it is particularly intended for acclerating wildlife surveys that use aerial images.
AIde is primarily developed by Benjamin Kellenberger, supported by the Microsoft AI for Earth program.
- Powerful: AIde explicitly integrates humans and AI models in an annotation loop.
- Fast: AIde has been designed with speed in mind, both in terms of computations and workflow.
- Flexible: The framework allows full customizability, from hyperparameters to models to annotation types to libraries. It provides:
- Support for image classification, point annotations, and bounding boxes (object detection)
- A number of AI models and Active Learning criteria built-in
- Interfaces for custom AI models and criteria, using any framework or library you want (see how to write your own model).
- Modular: AIde is separated into individual modules, each of which can be run on separate machines for scalability. It even supports on-the-fly addition of computational workers for computationally intensive model training!
You can try out the labeling frontend of AIde in a couple of demo instances:
- Image labels on the Snapshot Serengeti camera traps dataset
- Points on the VGG Penguins dataset
- Bounding boxes on the NOAA Arctic Seals aerial imagery
- Semantic segmentation on the Chesapeake Land Cover satellite imagery (early preview - stay tuned for official announcement of segmentation support!)
AIde consists of individual modules, organized as follows:
- LabelUI: responsible for delivering and accepting predictions and annotations to and from the user/labeler
- AIWorker: runs the AI model in the background to train and predict data
- AIController: distributes and manages jobs to and from the individual AIWorker instance(s)
- Database: stores all metadata (image paths, viewcounts, user annotations, model predictions, user account data, etc.)
- FileServer: provides image files to both the LabelUI and AIWorker instances
- Message Broker: AIde makes use of Celery, a distributed task queue leveraging message brokers like RabbitMQ or Redis.
The framework can be configured in two ways:
- As a static labeling tool (i.e., using only the modules in (a)). In this case there will be no AI assistance for labeling or prioritizing the relevant images.
- As a full suite with AI support, using all modules.
The individual modules need not be run on separate instances; it is possible to combine the components in any way and launch multiple (or all) modules on one machine. For example, the diagram above shows three AIWorker instances, but the number of workers can be chosen arbitrarily, and workers may be added or removed on-the-fly.
AIde ships with a set of built-in models that can be configured and customized for a number of tasks (image classification, object detection, etc.). See this page for instructions on how to use one of the built-in models.
AIde is fully modular and supports custom AI models, as long as they provide a Python interface and can handle the different annotation and prediction types appropriately. See here for instructions on using custom models.
See here for instructions on configuring an instance of AIde.
See here for instructions on launching an instance of AIde.
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.