[slijin]

The difference between brake and coast comes into play when the command being sent to the motors changes from some nonzero command to a zero command. When the command is nonzero, obviously, the motor will be spinning; changing and keeping the command at zero will eventually let the motor stop.

Brake and coast are two different ways to allow the motor to stop. If the motor controller is set to coast, when the command changes to zero, the motor controller simply uncouples the motor from the power, and lets the friction of the system bring the motor to a stop. When a motor controller is set to brake, however, it not only uncouples the motor from the power when the command changes to zero, but also shorts across the motor's terminals. This brings the motor to a stop more quickly because the motor, which is still spinning by itself, acts as a generator that, because its terminals are shorted together, now is powering itself to spin in a direction opposite the direction that it's spinning in.

For more information, look up how an H-bridge controller works.