Iterative Labelling & Model Building

[phorne-uncharted]

Currently, when building a model from user labelled data, a clone of the dataset retaining only labelled rows is used. This means that if the user is not happy with the results, they need to cycle back to the labelling screen, create a new clone, and then update the labels.

Instead of using the above approach, the user should be able to easily add more labelled data after reviewing the model results. The flow would be:

• User labels an initial set of data
• User clicks Create Model
• User views model results
• User chooses to label more data
• User clicks Create Model
• etc.

All of the above steps should be done without creating new clones for each iteration.

To get there, rather than clone the dataset (excluding rows missing labels), the client should instead model the label field and include a filter in the model creation request to remove unlabeled rows in the prefiltering step. Depending on the exact nature of that filter, there may be a slight change required to the API so that the client can specify the filter as used in prefiltering rather than as a prepend in the model creation pipeline.

That should let the user be able to iteratively specify labels to increase model quality.