[DonRotolo]

To expand on the 10k recommendation, pots with higher values will tend to be too noisy.

Also be sure you use a Linear (B) taper pot, not an audio (A) taper.
We have had success with cheap pots from Mouser, but the encoders RyanN mentioned are more stable.