[FEATURE] Training scripts for reproducing result of MobileNet V4 on IN1K.

[go2sea]

Thanks for adding mobilenet v4 to this great project. Timm is always the best place for sota image models.
I would like to ask if the training script of mnv4 can be released, because the training strategy is often difficult to synchronize.
I would like to compare mnv4 with my own models under timm training settings, specifically these two models. Thanks!

[rwightman]

https://gist.github.com/rwightman/f6705cb65c03daeebca8aa129b1b94ad

[hbellafkir]

Thanks alot

I'm trying to reproduce the results of the small version. I noticed a drop in accuracy after 5 epochs of warmup. This is the current state of the training:

('./output/train/20240904-133853-mobilenetv4_conv_small-224/checkpoint-2.pth.tar', 0.3519999998664856)
 ('./output/train/20240904-133853-mobilenetv4_conv_small-224/checkpoint-3.pth.tar', 0.28600000015258786)
 ('./output/train/20240904-133853-mobilenetv4_conv_small-224/checkpoint-4.pth.tar', 0.2579999999666214)
 ('./output/train/20240904-133853-mobilenetv4_conv_small-224/checkpoint-5.pth.tar', 0.23)
 ('./output/train/20240904-133853-mobilenetv4_conv_small-224/checkpoint-7.pth.tar', 0.1899999999666214)
 ('./output/train/20240904-133853-mobilenetv4_conv_small-224/checkpoint-6.pth.tar', 0.1839999999666214)
 ('./output/train/20240904-133853-mobilenetv4_conv_small-224/checkpoint-8.pth.tar', 0.168)
 ('./output/train/20240904-133853-mobilenetv4_conv_small-224/checkpoint-9.pth.tar', 0.15)
 ('./output/train/20240904-133853-mobilenetv4_conv_small-224/checkpoint-10.pth.tar', 0.142)
 ('./output/train/20240904-133853-mobilenetv4_conv_small-224/checkpoint-1.pth.tar', 0.1339999999666214)

Is this a normal behavior? I'm training using the script distributed_train.sh with 4 GPUs.

[rwightman]

@hbellafkir no, something is wrong there, did you use the --config argument to load the args from the yaml file in the gist or manually plug in the flags on command line? Might have missed something if the latter...

epoch,train_loss,eval_loss,eval_top1,eval_top5,lr
0,6.868316854749407,6.418820355529785,1.6240000033569335,5.63200001586914,0.00039999999999999996
1,6.473296778542655,5.7305817997741695,5.000000017089843,14.329999938964844,0.0007999999999999999
2,6.089241640908377,5.086933217163086,9.770000012207031,24.567999981689454,0.0012
3,5.716395105634417,4.514642635955811,15.673999989013671,35.11600002929688,0.0015999999999999999
4,5.41103070122855,4.030584475021362,21.852000041503906,44.270000041503906,0.002
5,5.127595356532505,3.6264143685150145,27.516000100097656,51.36400002197266,0.0019999991432636294
6,4.846096856253488,3.3142367100524903,32.242000032958984,57.16200005371094,0.001999996573055985
7,4.666107450212751,3.074735936431885,36.11200002685547,61.59599999267578,0.0019999922893814705
8,4.506485598427909,2.8880695221710204,39.43000004394531,64.74199998291016,0.001999986292247427
9,4.392298766544887,2.7388390077209475,42.126000090332035,67.32600003173827,0.0019999785816641295
10,4.250995227268764,2.615269951591492,44.35200006347656,69.33200002441406,0.00199996915764479
11,4.139863286699567,2.506826769981384,46.264000029296874,71.1299999633789,0.0019999580202055564
12,4.075272560119629,2.414468217277527,48.09200002441406,72.69200006591797,0.001999945169365512
13,4.012100458145142,2.3354783560180663,49.64800009765625,74.06399998535156,0.001999930605146677
14,3.9074611323220387,2.264585818786621,50.84199996826172,75.15800000976563,0.001999914327574007
15,3.910862718309675,2.2015090768432617,52.02999998779297,76.18600006103516,0.0019998963366753926
16,3.856656347002302,2.1477980322265626,53.14199996337891,76.96600005859375,0.0019998766324816606

[hbellafkir]

I used the --config argument. The problem was with the structure of the validation split. I downloaded ImageNet from Kaggle, where the validation split didn’t have a separate directory for each class. The accuracy makes more sense now after solving this. Thanks