Are you using the default camera code from Kevin Watson's
website? If not, you should try it with that so that you can tell if it's a problem with your software specifically. You should also try using the Labview CMUcam application (I don't have a link, but search for it on Chief Delphi) to see if the camera is locking on to something else, and to make sure that it is using the right tracking parameters, as well as to make sure that it's not a hardware problem.