What Is a Neural Network?

A neural network is a function that learns to map inputs to outputs by adjusting internal parameters called weights. It is composed of layers of artificial neurons. Each neuron computes a weighted sum of its inputs, adds a bias, and passes the result through a nonlinear activation function. By stacking layers and training with gradient descent, the network learns to approximate arbitrarily complex functions.

This playground trains a small network on a 2D classification problem. The input is a point (x, y); the output is a class (orange or blue). The decision boundary canvas shows how the network partitions the plane. Press Train and watch the boundary evolve as the network learns.

How It Works

This network uses standard backpropagation with stochastic gradient descent. At each epoch, it makes a forward pass (computes predictions), measures the loss (binary cross-entropy), then makes a backward pass (computes gradients via the chain rule), and updates weights in the direction that reduces the loss. The learning rate controls how large each update step is.

The Architecture

The input layer has 2 neurons (x and y coordinates). The hidden layers have the number of neurons you select. The output layer has 1 neuron with a sigmoid activation (producing a probability between 0 and 1). The activation function in the hidden layers controls how the network bends the decision boundary:

• ReLU (Rectified Linear Unit): max(0, x). Produces piecewise-linear decision boundaries. Fast to train, can "die" (permanently output 0 if all inputs are negative).
• Tanh: Smooth S-curve from −1 to 1. Produces smooth, curved boundaries. Can saturate (gradients vanish for large inputs).
• Sigmoid: Smooth S-curve from 0 to 1. Similar to tanh but historically important. Slower to train due to vanishing gradients.

What to Try

• Circle dataset, 1 hidden layer, 2 neurons: The network cannot learn a circle with only 2 neurons. Increase to 4 or 6 and it works. This demonstrates the importance of width.
• XOR dataset: A single-layer network cannot solve XOR (this is the famous result from Minsky & Papert, 1969). Add a hidden layer and it learns immediately.
• Spiral dataset: The hardest problem here. Requires 2–3 hidden layers with 6+ neurons and patience. Watch the decision boundary slowly twist into shape.
• ReLU vs. Tanh: Try both on the circle dataset. ReLU produces angular boundaries (piecewise linear); tanh produces smooth curves.
• High noise: Set noise to 0.3+. The network must learn to generalize rather than memorize. Too few neurons = underfitting. Too many = overfitting (the boundary wraps around individual noisy points).
• Learning rate: Try 0.001 (very slow convergence) vs. 0.1 (fast but unstable—loss may oscillate).