- How to train a Neural Network using MNIST database with plain numpy
- How to use the trained Neural Network to classify digits from input images
- Stochastic Gradient Descent (SGD) for training the network (optimizing the network weights)
- Backpropagation for calculating the gradients that SGD needs for adjusting the weights of the networks
- Tensors (n dimensional matrices) - the weights of the Neural Network
- In the training phase the program is using MNIST database in Python pickle format
- In the classifying phase the program is getting 28x28 pixel grayscale images representing digits as inputs and the program is supposed to output the actual digit the image contains
- Steve Renals - Stochastic gradient descent; classification
- Chris Williams - Optimization
- Justin Johnson - Backpropagation for a Linear Layer
- ML Cheatsheet - Backpropagation
- PyTorch documentation
- Peter Sadowski - Notes on Backpropagation
- Michel Nielsen - Neural Networks and Deep Learning Book
- Terence Parr & Jeremy Howard - The Matrix Calculus You Need For Deep Learning
- Neural Networks for Machine Learning by Geoffrey Hinton
- CS231n: Convolutional Neural Networks for Visual Recognition by Andrej Karpathy
- Coursera Machine Learning by Andrew Ng
- Practical Deep Learning for Coders Part 1 by Jeremy Howard
- Khan Academy - Multivariable Calculus by Grant Sanderson
- Deep Learning by Grand Sanderson