A two-layer network in plain NumPy that reads handwritten digits, with every gradient written by you.
Load scikit-learn's digits dataset (8 by 8 images of 0 to 9) and split it 80/20 into training and test sets.
Write the forward pass: 64 inputs, 32 hidden units with ReLU, 10 outputs through softmax.
Write the cross-entropy loss and the backward pass by hand, then check one gradient against a numerical estimate.
Train with mini-batch gradient descent and plot training and test loss on the same chart.
Show the ten test digits it gets wrong with the most confidence, and say what they have in common.
How you'll know it worksTest accuracy of at least 95%, and your hand-written gradients agree with the numerical ones to within 1e-5.
Python, NumPy, scikit-learn (for the data only) and matplotlib. Runs on any laptop in seconds. Uses lessons 01 to 10.
Project: A digit reader from scratch
From insideout, Neural Networks: A two-layer network in plain NumPy that reads handwritten digits, with every gradient written by you.
Steps:
1. Load scikit-learn's digits dataset (8 by 8 images of 0 to 9) and split it 80/20 into training and test sets.
2. Write the forward pass: 64 inputs, 32 hidden units with ReLU, 10 outputs through softmax.
3. Write the cross-entropy loss and the backward pass by hand, then check one gradient against a numerical estimate.
4. Train with mini-batch gradient descent and plot training and test loss on the same chart.
5. Show the ten test digits it gets wrong with the most confidence, and say what they have in common.
How you'll know it works: Test accuracy of at least 95%, and your hand-written gradients agree with the numerical ones to within 1e-5.
Runs on: Python, NumPy, scikit-learn (for the data only) and matplotlib. Runs on any laptop in seconds.
Help me build this on my machine, one step at a time. Ask what I've tried first, give hints before full answers, and check my work against "How you'll know it works".