Quote:
Originally Posted by Greg McKaskle
I still fail to see how the FMS could provoke this. It is more likely that the shiny field or heat of the match are contributing. Your logs show your lag and it is not bouncing around causing timing glitches.
If it was your job, or your programming test, to find this bug and explain the failure, what would you do?
Greg McKaskle
|
Sorry, if I wasn't clear. I see no reason to think that FMS or the field are the problem. I think the problem is somewhere on the Robot. However, all of our testing so far only triggers while on the field. So a forensic analysis tries to understand what possible difference there could be on the field that would trigger a problem. And I agree, the best suspect is the target rich environment. But as you noticed, the camera threads all appear to stay alive, and the main loop code has very little overlap with camera information. It's hard to connect those dots. The difference in latency or bandwidth would appear to be an unlikely trigger (you have to imagine that 2 ms versus 1 ms makes a difference), but I have no other theories, so I mention it.
I hope to run a test to see if generating error messages at the same time Network Tables traffic is being sent can cause a crash; that would test my first hunch. The other step I would take is to put wireshark on while it operates to see if that generates any further clues.
If I had infinite time, I'd take the whole WPILib stack and run it under a tool like Valgrind while also operating our robot to see if I could shake free any bugs in that code. That Network tables code was already the culprit in a nasty set of crashes once this season, and it is not trusted by other mentors I've spoken with. It's a sufficiently complex body of code that simple inspection isn't enough to identify any problems.
I don't know enough about the VxWorks environment to know if there is a Valgrind like tool available on the Robot.
Beyond that, though, without the ability to reproduce the problem, it becomes difficult to solve the issue.
Also, note that our robot is done and has no further need for this capability. I'm now just trying to solve the puzzle for my own edification, and for the possibility that it might help another team.
Cheers,
Jeremy