Another reason for fine threads, aside from the high clamping force, is their ability to be used for fine adjusts on mechanesms. For instance, on a machine a buddy and I made at work to load boards from a casette onto a conveyor, we used a simple threaded hole and a fine threaded socket head bolt with 3B threads so theyre sticky (grab the bolt a little) to finely adjust the pitch at which the gripper and board intersect.
Also, fine threads are often more and more useful as screws get smaller since they increase the mechanical advantage of the screw. Basically, the finer the thread is (ignoring friction) the less input torque on the bolt head it takes to get the same compression on the material. So on a small screw (like the machinemaker's favorite, the 5-40) the shaft of the screw is so thin that very little torque can be put on it. So, the high thread count makes up for it.
On the standard sizes... I think its just manufacurers of certain devices like pneumatics and such had to standardize on something, and then machine designers had to use these components, so over the years the standardization has filtered down through industry so now we all know the famous 1/4-20s, 8-32s, 10-24s
, etc.
just my 2¢.
-q