Alon Oring: A Shallow Introduction to Self-Attention | PyData Tel Aviv 2024

[kuberiitb]

Video: https://www.youtube.com/watch?v=HAKgbLcSiVg

0:00: AI and Job Automation Concerns
1:33: Speaker Introduction and Background
3:38: Basics of Neural Networks and Perceptrons
5:20: Recurrent Neural Networks (RNNs) Overview
8:17: Introduction to Word Embeddings in Self Attention
10:50: Self Attention Mechanism and Weighted Average
15:53: Detailed Explanation of Self Attention with QKV Vectors
18:19: Understanding Self Attention and its Advantages
21:15: Transformer Architecture and GPT Model Overview
22:42: GPT Training and Fine-Tuning Process