[Jared Russell]

Do you have access to an oscilloscope? I would hook the pot up to that and watch the wiper voltage change as you turn it. If you see a delay there, you probably have a wiring issue or a busted pot.

Also, you can verify that you have the right connections. One end should be +5V, the other end should be GND. The middle (the wiper) is where you are making your reading.

Lastly, what is the resistance of the pot? You may have an RC-type delay somewhere in the circuit.