[FEATURE]: Support Optional Train and Test Transforms for Image Training Endpoint

[codingwithsurya]

Feature Name

Optional Train and Test Transforms for Image Training Endpoint in Django Migration

Your Name

Surya Subramanian

Description

After migrating our image training endpoint from Flask Blueprint to Django, we identified the need for supporting train and test transforms. This feature should be available for user-specified datasets in the frontend, and these parameters should be optional.

Transforms are methods applied to the input data before it is fed into the model for training or testing. These methods include normalization, augmentation (like rotation, translation, flipping), and others, depending on the specific needs of the dataset and model.

Train Transforms: Applied to the dataset during the training phase. They are generally used for data augmentation to introduce variety in the dataset, which helps prevent overfitting and improves the model's generalization.

Test Transforms: Applied to the dataset during the testing or validation phase. These are usually lighter and consistent, mainly used for preparing the data in a specific format or scale that the model expects.

You can implement these as JSON Objects.

Example :

{
  "train_transforms": [
    {"type": "Resize", "parameters": {"size": 256}},
    {"type": "RandomCrop", "parameters": {"size": 224}},
    {"type": "ToTensor"},
    {"type": "Normalize", "parameters": {"mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225]}}
  ],
  "test_transforms": [
    {"type": "Resize", "parameters": {"size": 256}},
    {"type": "CenterCrop", "parameters": {"size": 224}},
    {"type": "ToTensor"},
    {"type": "Normalize", "parameters": {"mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225]}}
  ]
}

Some useful info on where to get started :

This is just some initial thoughts on how i would try to get started on this:

Backend

• In the backend, you can to modify the imageTrain function in the image.py file to accept the transforms as parameters and apply them to the data.
• You can add the transforms as fields in the ImageParams object. The transforms should be JSON objects, so you can parse them using json.loads().
• Then, you need to apply the transforms to the data before feeding it into the model. The exact way to do this depends on the specific transforms and the data format you decide, but it generally involves calling a function or method with the data and the transforms as arguments.

Here is an example of how you can modify the imageTrain function:

def imageTrain(request: HttpRequest, imageParams: ImageParams):
    train_transforms = json.loads(imageParams.train_transforms)
    test_transforms = json.loads(imageParams.test_transforms)

    # Apply the transforms to the data
    train_data = apply_transforms(train_data, train_transforms)
    test_data = apply_transforms(test_data, test_transforms)

    # Continue with the rest of the function

In this example, apply_transforms is a function that applies the transforms to the data. You need to implement this function according to your specific needs.

Frontend

In the frontend, you need to add fields for users to specify the transforms in the UI. This can be done in the ImageParametersStep.tsx file.

You already have TrainTransformComponent and TestTransformComponent components for handling the transformations. You can add fields in these components for users to specify the transforms.

Some helpful file-finding info:

In the frontend, The test and train transforms are located in the ImageParametersStep.tsx file in the frontend codebase. The TrainTransformComponent and TestTransformComponent are the components that handle the transformations for the training and testing data respectively.

In the backend, our image params are under ...image/schemas.py and our image training is under ...image/image.py

Requirements
Optional Parameters for Transforms: The frontend should allow users to specify train and test transforms for their datasets. These parameters should be optional to maintain flexibility for users who may not need this feature.

API Adaptation: The API needs to be capable of ingesting train and test transforms, not only for user-defined datasets but also for default datasets. This ensures consistency and flexibility across different types of dataset inputs.

Backward Compatibility: Ensure that the changes are backward compatible with the existing system and do not disrupt current functionalities.

Testing: Implement testing via Postman + MNIST or CIFAR dataset to make sure the endpoint works well with the new changes.

[github-actions]

Hello @codingwithsurya! Thank you for submitting the Feature Request Form. We appreciate your contribution. 👋

We will look into it and provide a response as soon as possible.

To work on this feature request, you can follow these branch setup instructions:

1. Checkout the main branch:

```
 git checkout nextjs
```

2. Pull the latest changes from the remote main branch:

```
 git pull origin nextjs
```

3. Create a new branch specific to this feature request using the issue number:

```
 git checkout -b feature-1057
```

Feel free to make the necessary changes in this branch and submit a pull request when you're ready.

Best regards,
Deep Learning Playground (DLP) Team

[dwu359]

Can helpful if more details are given about what train & test transforms are in terms of swe terms (object structure of a train and test transform, effects on training, etc)

[codingwithsurya]

Can helpful if more details are given about what train & test transforms are in terms of swe terms (object structure of a train and test transform, effects on training, etc)

hmm i think i see what your saying but could you be more specific? Just want to make sure I address everything correctly.

[karkir0003]

@codingwithsurya , I think @dwu359 meant an example of what a valid sequence of train and test transforms can look like. Like, what will the datatype be?

[codingwithsurya]

@codingwithsurya , I think @dwu359 meant an example of what a valid sequence of train and test transforms can look like. Like, what will the datatype be?

got it. Just updated the issue with more specific info. Please take a look @karkir0003 .

Let me know if you have any other questions @Keon-San

[karkir0003]

lgtm @codingwithsurya

[karkir0003]

for the json thingy, this lines up with how the other api endpoints parse the params?

[codingwithsurya]

for the json thingy, this lines up with how the other api endpoints parse the params?

not sure about that. I just gave a general example, but hope that gives you an idea @Keon-San