Nn from scratch 🕸️ (#10)

* fix formatting

* coding a neural network from scratch

content/posts/2024-06-06-neural-network-from-scratch/index.mdx

@@ -0,0 +1,51 @@
+---
+title: Coding a neural network from scratch
+author: Sudaraka Jayathilaka
+date: 2024-06-06
+hero: ./images/hero.png
+excerpt: Learning how AI works on a deeper level has always been in my bucket list. I found this amazing video by
+---
+
+Learning how AI works on a deeper level has always been in my bucket list. I found this amazing video by [Andrej Karpathy](https://x.com/karpathy)
+which walks you through the process of creating an autograd engine first and then move on to create a very basic Neural Network. So here is my
+experience following the video.
+
+[](https://youtu.be/VMj-3S1tku0?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ)
+
+## Autograd Engine
+
+![](./images/nn.png)
+[image source](https://cs231n.github.io/neural-networks-1/)
+
+A Neural Network is basically a graph. once we give an input, the neural network returns an output. Each of these neurons have certain parameters
+which impacts the output directly. We train neural networks to output a desired value by tweaking these parameters in neurons. But how can we derive
+the parameter values that could cause the neural network to output the desired value?. 
+
+Turns out, old friend calculus comes to help here. We can approximate the impact of a certain parameter on the output of the Neural Network, by
+calculating the gradient of the parameter with respect to the output. 
+
+Autograd engine helps us to build a Neural Network (modeled as a graph), and easily do backpropagation to calculate the gradients of each parameter.
+Hence the name Autograd Engine. 
+
+## Tiny Grad
+
+[](https://github.com/sudaraka94/tinygrad)
+
+Tiny grad is the engine I build. This is a very basic Auto Grad engine which
+- Supports implementing Simple Neural networks
+- Only works with scalaer inputs
+- Supports backpropagating through the neural network and calculating gradient values for each parameter
+
+I did a tiny experiment with tinygrad, where I created a simple neural netowrk and trained it to predict a given
+input. I was simply playing with a set of numbers, but it was quite fun. You can find my jupyter notebook below,
+
+[](https://github.com/sudaraka94/tinygrad/blob/main/tinygrad.ipynb)
+
+## Micrograd
+
+The tinygrad engine is mostly inspired by Andrej Karpathi's Micrograd, Autograd Engine.
+
+[](https://github.com/karpathy/micrograd)
+
+
+[article banner source](https://miro.medium.com/v2/resize:fit:700/0*AONVmd3v4wO_dWr6)