If you can't be sure that the motor will be moving in the same direction it's commanded, there's no good way to do what you want in software. The best thing I can think of is to use two gear tooth sensors on the same gear, mounted so that they give a quadrature signal just like a rotary encoder would.
Or just add a rotary encoder and be done with it.
