insideout Get Pro

10 · Train a classifier on your own autograd

Pro

Neural Networks, lesson 10

Train a classifier on your own autograd

Put the pieces together and train a small network end to end.

"Autograd you can step through" (the Value class) and "Loss functions" (cross-entropy). "Diagnosing a network that won't train" helps with the last section.

Part 1. Two moons

This lesson puts the whole module into one run: a small network learns to tell two groups of points apart, trained with the Value class from "Autograd you can step through".

The points come from two half-circles hooked into each other, like two crescent moons. Each point has two numbers, its x and y position, and a label: 1 for the upper moon, 0 for the lower one. A little random noise blurs the edges.

Example 1.1

Making the data

  1. t=a random angle from 0 to π

    Pick a random spot along a half-circle. π (about 3.14) is half a turn.

  2. upper moon(cos t, sin t)label 1
    lower moon(1 − cos t, 0.5 − sin t)label 0

    The upper moon arcs over the top. The lower one is the same arc flipped and shifted, so each moon's tip pokes into the other's hollow.

  3. x, y+noise with spread 0.1

    Blur each point a little, so the two moons nearly touch.

  4. 80 points→training
    200 points→testing

    The network learns from 80 points. Another 200, made the same way, are kept aside to measure it.

Test set

Examples kept out of training and used only to measure. A network can score perfectly on the examples it trained on and still fail on new ones. Only unseen examples show whether it learned the shape.

Your turn

A network whose boundary can only be a straight line trains on the moons. What do you expect?

The moons hook into each other, so no straight line splits them: the XOR problem from "Stacking neurons into layers" again. A straight cut still gets most points right. With a single hidden neuron, whose boundary is a straight line, this data stalls at 89% of the test points.

Part 2. The network and its loss

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come
  1. The network and its loss
  2. One training step
  3. Reading the run
See Pro plansI have Pro: sign in
The last lesson in Neural NetworksBack to the overviewHow a network learns, built from scratch Back to Neural Networks

Neural Networks overviewSpotted a mistake? Tell us.