Connecting Neurons: Building a Neural Network
See why networks need hidden layers, and how stacking layers makes them deep.
One neuron on its own can only split the world with a straight line: "yes" on one side, "no" on the other. Many real patterns are not straight. In the simulation, the dots inside the circle are one group and the dots outside are another. No straight line can wrap around a circle.
The fix is a hidden layer: a layer of neurons between the inputs and the output. Each hidden neuron draws its own straight line, and the output neuron combines their answers. A few straight lines together can surround a shape; more lines make it smoother.
Stack several hidden layers and you get a deep neural network. Each layer builds on the one before: in a photo network the first layers spot edges, the next ones shapes, and the last ones whole faces. More neurons and layers mean more parameters, and today's biggest networks have billions of them.
Key takeaways
- A single neuron can only separate things with a straight line.
- Hidden neurons each draw a line; the next layer combines them into curved shapes.
- Deep networks stack many layers, each building more complex ideas from the one before.
- Every neuron adds weights, so bigger networks have many more parameters to learn and compute.
Check your understanding
Go deeper (more technical)
In the simulation the weights of the hidden neurons were chosen by hand so you can see their lines. In a real network, gradient descent finds them, and the lines become soft curves thanks to the squish functions.
A famous result (the "universal approximation theorem", 1989) says that one hidden layer with enough neurons can approximate almost any pattern. In practice, deep networks with many smaller layers learn far more efficiently than one enormous layer, which is why today's models are deep.