We encountered a problem for the first time today at the regional. It only happens in autonomous mode. Take the robot to the playing field, plug in the backup battery, power on the main battery, press reset on the controller, the camera does not initialize.
I’ve read other people have encountered the same problem but have not seen anyone have an actual fix.
The camera code is correctly called from autonomous mode and works if the robot is put into user mode then back into autonomous mode (which you obvisously cannot do on the field). We even did this in the pits and left the robot powered on till it was put on the playing field. It worked that time.
We have since found a spot in the code which is called in autonomous mode right before the inner while loop. We are going to try to reset the camera intialization variable there and test it during the finals.
The behavior is the same as if a controller reset does not call the program’s static initializers in autonomous mode!
We have had some weird problems on the field all day too but we COMPLETELY unplug all our batteries and then check all our electrical connections and put it back in. It seemed to have worked fine after that.
We would completely unplug our batteries also. We could reproduce the problems in the pits by setting our OI to autonomous. During the matches today the camera did initialize so resetting the variable I mentioned does work.
Hmm…
You know that section in the User_Autonomous_Code function, right before the “while (autonomous_mode)” loop (and right after all of the PWM values are reset)?
Our team had been experiencing problems with that, and so (in my hacking nature…) I had put the Initialize_Camera function in that section. I don’t know why, but it helped, and now the camera always initializes when Autonomous mode starts (or is invoked).
also, while you are executing the Autonomous loop, are you sure you are calling the function “Camera_Handler()”?
Yes. Our programmer found camera searching was not working in autonomous and he and I tracked down why and added both “Camera_Handler()” and “Servo_Track()”.
The behavior I described as if the static initializers are not being called under certain conditions is also happening in the recent thread The 8.2 (or 8.3) Battery Voltage Bug.
Come to think of it, we had “intermittent” camera initialization problems from day one. Most of the time we assumed it was from a low backup battery so we were constantly swapping those in and out. Camera initialization always failed for autonomous mode when the competition port was connected. We struggled a long time with assuming we were not starting up the robot the same way every time. It was not until Friday night at the regional when we were finally able to consistently reproduce the problem in the pits. That was when we decided to try a non-static initializer.
And I no longer have access to the robot to write a test to prove the static initialization conditions.
The static initializater for camera initialization is in Kevin’s bells and whistles camera code.
And I no longer have access to the robot to write a test to prove the static initialization conditions.
I do, I have our teams 2006 RC sitting in front of me (since I’m going to send it to IFI this week). If I have a spare moment I’m going to write some test code to check on this, as well as read over the 8722 manual and errata (I had a similar issue to this with another 18F chip on a project at work, so I wouldn’t be surprised).
Oy, amazing how repeated testing of a bot at home can show things to be surprising robust, but fail completely at competition like clockwork…
It seems that the problem can manifest itself as a messed up state machine - where there is the large switch() statement with CASE_ONE through CASE_WHATEVER, the state becomes set to zero accidentally so it falls through every time and does nothing.
I got to thinking about this. Is this the reason IFI has the second “initialization” section directly before the autonomous while loop? Would it be a good idea just to bypass IFI’s initialization calls altogether and make your own initialized / not initialized flag that, when zero, calls the init routine and sets the flag to 1?
I distinctly recall a similar problem in our custom code last year, with the robot acting completely bezerk on the field, yet in the pit with the dongle, it worked perfectly. I had heard a rumor about the Arena Controller forcing some king of sync with the OI and RC; is there anything to justify this rumor? It might explain a lot of the “it worked perfectly until I put it on the field” problems.
Would this issue be reproducable on a 2004/2005 RC, since it seems to be a problem with IFI’s default code?
Neither our code nor tracking.c uses switch statements. Camera.c does though and does not have default cases. Bad idea. I *never * write switch statements without a default case even if it just prints an error message.
The only initialization I see is giving all outputs a default value. Not only is there a comment explaining why, but the user code gets called once on startup even when in autonomous mode. So forcing outputs to off is a good idea.
Thanks for the ideas. In the rush at the Regional we never had the time to sit down, develop a test plan, and methodically track down the problem. But we did find a sequence of events where we could reproduce the problem in the pits. Our code was just not instrumented to diagnose the problem.
Today I developed a test plan involving: using the default frc_camera_2 code, adding camera tracking to autonomous, and instrumenting the camera and tracking initialization variables to show their state on OI lights. My team contact is on the other coast for 2 weeks so I may not be able to arrange test time for a while.
Unfortunately our regional is over and we are done for the year. Our strategy was heavily weighted towards scoring during autonomous. All the pieces were in place but the camera never worked during autonomous.
In general, if you end up with an invalid state you will have unpredictable results. Specifically with regards to camera.c, I have looked at the code but cannot predict what might happen. The first walk through looks like it should work. But I have not worked through the possibilities to see if it is possible to have an uninitialized state.
Sure, on large, complex systems with lots of resources you’d instrument the code or HDL to catch such cases. Instrumenting a very simple state machine that cannot (by inspection) have an invalid state, in a resource constrained system where code readability is of paramount importance, doesn’t make sense to me. If the static state variable isn’t getting initialized at boot time, a wonky state machine is the least of your worries :).
Now I know what that third LED relay light is for, we found a use for
all of the other lights on the OI. We will label it the “totally
screwed” light. It is best to know this condition as early as possible.
Sounds like the default action for the switch statement to me…
For the record, I was talking specifically about the camera communications state machine - when I inserted a default to catch the improper ‘zero’ state, it printed out errors galore.
After working with many experienced software engineers over the years I can confirm that many experienced software engineers have that same opinion. Yet just the camera.c and tracking.c files have 11 debug statements. One of those is implemented in a state machine and is called 18 times.
For all the high school programmers watching out there, creating robust software systems is an important consideration in software design, even in a resource limited machine. I have programmed on much smaller systems than the IFI. Just because you think certain conditions should not happen does not mean that they never will.
I know. :rolleyes: I will persue it until I can track down the error we observed.
Our strategy was heavily weighted towards scoring during autonomous. All the pieces were in place but the camera never worked during autonomous.