[Jared Russell]

The small circles are neurons - in artificial neural networks, neurons have N inputs and a single output. Each of the N inputs has an associated weight - the output of the neuron is the weighted sum of the each input multiplied by its weight. There are many variations on this scheme, but this is a general starting point for a neural network.

However, what is pictured needs some more explanation to be meaningful. This is a picture of a generic one-layer artificial neural network. Neural networks essentially try to solve nonlinear optimization problems - what is the problem (cost function to be minimized) in this case?