The Rise and History of Neural Networks

---

Timeline and Article:

• 1943: McCulloch & Pitts - A logical calculus of the ideas immanent in nervous activity
• 1950s: Hubel & Wiesel - Cat Experiment & Basis of Visual Perception
• 1958: Rosenblatt - The Perceptron: A Probabalistic Model For Information Storage And Organization In The Brain
• 1980: Fukushima - Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position
• 1986: Hinton, Rumelhart, & Williams - Learning Representations by back-propogating errors
• 1989: LeCun & Friends - Backpropogation Applied to Handwritten Zip Code Recognition
• 2001: Viola & Jones - Rapid Object Detection using a Boosted Cascade of Simple Features
• 2006: Hinton, Osindero, & Teh - A fast learning algorithm for deep belief nets
• 2011: Glorot, Bordes, & Bengio - Deep Sparse Rectifier Neural Networks
• 2014: Krizhevsky, Sutskever, Hinton - ImageNet Classification with Deep Convolutional Neural Networks

---

Other Articles that may also be of interest:

• The inside story of how AI got good enough to dominate Silicon Valley

---

Demo Code Implementations from Jon Krohn

• LeNet-5 Keras Implementation
• AlexNet Keras Implementation