[EricH]

Depends on how you're off by those few thousandths. If the axle is too thick, not much (besides the fact of tolerances).

If it's out of round or the hole is off, that's a little worse, but not a lot.

I'd like you to meet a machinist's best friend: tolerances. A few thousandths could easily be within tolerance and have almost no effect.

A few thousandths isn't much to worry about. A few tens of thousandths would be.