Extremely slow but mostly functional subset of pytorch, with an autodiff engine and small nn library. Inspired by Karpathy's micrograd but supporting tensor-valued nodes and some more operations like broadcasting, matrix multiplication and convolutions. See demos/mnist.ipynb for example usage
- Karpathys video on autograd
- Course on autograd
- Colab example implementation of pytorch autograd
- Articles on optimization algorithms, other good articles on same site (d2l.ai)