PV021 Neural Networks

Deep Learning from Scratch

Feed Forward Neural Network without the use of advanced libraries or frameworks.

Running

cargo run --release

Example logs

Epoch # 0/5  | Batch #  1/199 | Loss 2.4956 | Learning rate 0.00099900 | Step time 191 ms
Epoch # 0/5  | Batch # 21/199 | Loss 0.6277 | Learning rate 0.00097943 | Step time 192 ms
Epoch # 0/5  | Batch # 41/199 | Loss 0.5979 | Learning rate 0.00096061 | Step time 197 ms
Epoch # 0/5  | Batch # 61/199 | Loss 0.5577 | Learning rate 0.00094251 | Step time 188 ms
Epoch # 0/5  | Batch # 81/199 | Loss 0.5354 | Learning rate 0.00092507 | Step time 197 ms
Epoch # 0/5  | Batch #101/199 | Loss 0.5134 | Learning rate 0.00090827 | Step time 187 ms
Epoch # 0/5  | Batch #121/199 | Loss 0.5106 | Learning rate 0.00089206 | Step time 189 ms
Epoch # 0/5  | Batch #141/199 | Loss 0.4118 | Learning rate 0.00087642 | Step time 207 ms
Epoch # 0/5  | Batch #161/199 | Loss 0.5314 | Learning rate 0.00086133 | Step time 182 ms
Epoch # 0/5  | Batch #181/199 | Loss 0.4518 | Learning rate 0.00084674 | Step time 192 ms
Epoch # 0/5  | Batch #199/199 | Loss 0.4496 | Learning rate 0.00083403 | Step time 199 ms
############
Validation metrics for epoch #1/5
Evaluating on Valid split
Got 7633 correct in 9000 examples
{
  6: 488,
  4: 617,
  5: 776,
  1: 881,
  9: 879,
  3: 802,
  7: 819,
  8: 852,
  2: 732,
  0: 787,
}
Valid accuracy: 0.8481
Epoch time: 42 sec

DATASET

Fashion MNIST (https://arxiv.org/pdf/1708.07747.pdf) a modern version of a well-known MNIST (http://yann.lecun.com/exdb/mnist/). It is a dataset of Zalando's article images ‒ consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. The dataset is in CSV format. There are four data files included:

• fashion_mnist_train_vectors.csv - training input vectors
• fashion_mnist_test_vectors.csv - testing input vectors
• fashion_mnist_train_labels.csv - training labels
• fashion_mnist_test_labels.csv - testing labels