This course is intended as a quick introduction of fundamental concepts of deep learning, covering neural network basics, training methods, as well as a few examples of specific applications such as convolutional neural networks for computer vision and the transformer model for natural language processing.
More specifically, the following topics will be covered:
- Fundamentals of neural networks
- Training deep learning models: stochastic gradient decent, optimizers, loss functions, regularization, etc.
- Convolutional Neural Networks (CNNs) for computer vision: basics of CNNs, image classification and generation
- Natural Language Processing (NLP) with transformers: basics of NLP, the transformer model, attention mechanism, etc.
The lessons will blend theory with hands-on practice, using PyTorch for practical exercises. We will run these sessions on the Piz Daint supercomputer at CSCS.