PyTorch conversion

[mvsjober]

• [EASY] Exercise 1: 01-tf2-test-setup.ipynb
• [YES, DOUBLE CHECK] Exercise 2: 02-tf2-mnist-mlp.ipynb -> pytorch-mnist-mlp.ipynb
• [EASY] Exercise 3: 03-tf2-mnist-cnn.ipynb
• [NON-TRIVIAL] Exercise 4: 04-tf2-imdb-rnn.ipynb
• [YES, DOUBLE CHECK] Exercise 5: Task 1 & 2 (yes, except BiT and TFRecord ones)
• [YES, DOUBLE CHECK] Exercise 5 / Extracurricular 4: ViT
• [NON-TRIVIAL] Exercise 6 (diff models)
• [YES, DOUBLE CHECK] Exercise 7 (20ng, rnn, cnn, bert)
• [NON-TRIVIAL] Exercise 8 (aclImdb-gpt)
• [EASY] Exercise 9 (multi-gpu and Horovod) -> multi-gpu and DDP

Notes of things still to do:

• Need to add pytorch_test.py
• check day2 pytorch scripts that style is similar to day1 (use Sequential style for simple, check training and testing loops)
• pytorch VIT is basically broken, needs to be looked at
• 20ng scripts OK, but RNN (71% vs 85%) and CNN (73% vs 94%) have worse accuracy results
• Try find something similiar to aclImdb-gpt but for PyTorch

[jmakoske]

Some modern CNN, not necessarily BiT, could be nice to show that CNNs can compete

[mvsjober]

Do we still keep the show_failures and confusion matrices and things in the mnist-mlp?

[jmakoske]

Do we still keep the show_failures and confusion matrices and things in the mnist-mlp?

Maybe not, or move those to an "optional" subsection or something, as there are more things to introduce in the pytorch code?