[FTC]: Playing Forensic Roboticist.

[emmell]

Hmmmm.... I wonder if ROBOTC users see this as well.

NI/PITSCO - please notice this and investigate.

[ssi]

Phil,

I can't help you with checking available memory while running, but we have seen file errors before. However, all of these could be traced back to coding errors. At the moment, I cannot remember exactly the problem, but I thought it had to do with attempting to talk to a motor or servo controller on the wrong port (or wrong place in the daisy chain). At the time we were probably using NXT-G (but could have been LabView, we went through all 3 packages before settling on RobotC). Is it possible that you have a coding problem in a section of code that is only run on rare occasions possibly due to the position of the robot at the end of autonomous/beginning of Teleop.

As for the frustration of the robot not responing to controls, our team managed to experience that twice at the MD competition. The first time it was caused by a team member forgetting to plug in the external battery in when we swapped for a fresh one. Fortunately our team mate won that qualifying match for us. The second time was in the semi-finals when the RobotC software was not detecting the external battery and refused to run. After the match, we unplugged and re-plugged the battery and it registered normal!?!

We competed 1 week later at the PA tournament and had no control or connection problems with our robot (we also reset bluetooth between all matches). However in our teams second semi-final match at least 3 robots locked up (though one may have been broken) just as our team mate was about to score 2 racks. This would almost have to been an FMS or USB problem.

David

PS. Good luck at Atlanta

[ptan]

Testing on RobotC:

So far I have not yet been able to recreate Phil's FMS problem, nor have I seen the File Error problem to date. We are using RobotC, so that may be a factor in the system. I have experienced bluetooth problems when the NXTs auto power down.

I will keep on testing to see if I can recreate the problem.

Paul Tan.

[Rick TYler]

Mysteriously, software errors have not been our biggest problem, although we did lose a pair of drive motors when a robot went into "drive full speed forever" in autonomous and the referees wouldn't let our drive team retrieve the robot before the motors released their magic smoke. This affected all four robots in the match, although only two smoked their motors. In one other incident where it took the FMS about 15 minutes to talk to one of our robots (that had happily talked to the same controller an hour before), our on-field software incidents have been very few. We use RobotC.

[PhilBot]

Originally Posted by Rick TYler

although we did lose a pair of drive motors when a robot went into "drive full speed forever" in autonomous

Your team name seems well earned

..... Exothermic Robotics Club

[Rick TYler]

Originally Posted by PhilBot

Your team name seems well earned

..... Exothermic Robotics Club

Well, as the students say, it was on fire when we got there.

[PhilBot]

Originally Posted by ptan

Testing on RobotC:
So far I have not yet been able to recreate Phil's FMS problem, nor have I seen the File Error problem to date. We are using RobotC, so that may be a factor in the system. I have experienced bluetooth problems when the NXTs auto power down.
Paul Tan.

That's good news because it probably means that my "fault" is not systemic. It may be localized just to complex LabVIEW code.

With only my "gut" to go on, I'm leaning towards the conclusion that the NXT Firmware that's required for LabVIEW has some sort of memory leak or other error that causes the program to terminate if it's been run multiple times without being re-booted.

It also seems that downloading a new program can cause simmilar issues unless the NXT is rebooted before running. I've seen this a few times.

It's quite possible that this problem is agravated by the complexity of our code in general, but my experience from testing today, is that if the NXT is rebooted EVERY TIME before a match, I never see the Program Abort (which seems to be due to the "File Error" problem).

My only concern is that if we have to wait too long before actually beginning Auto, the same problem may materialize. I only hope that the memory loss (or whatever) is due to something like memory not being freed up when the program ends..... Thus the first run should always be safe.

.. Some other stuff I've determined... that may only apply to LabVIEW

It seems llike the generic driving "Glitches" that we've been experiencing may come from incomplete motor commands reaching the controller. The Hitechnic motor controller is not able to tell when a full message has been received, so if only some of the bytes arrive, the controller acts on them anyway. If these bytes are part of a "Specify Position" type command, then the command is only sent once and consequently if it's incomplete, the robot will go to an incorrect encoder destination.

Instead, by continually resending the desired command, I've been able to completely eliminate any erratic drive actions. I split the "Specify Position" VI into two parts. One part to set the destination, and the other part to check if it's reached it yet. I can now call them in a loop without "blocking", to reinforce any incomplete command transfers. By intermixing both Wheel and Arm commands I can command both systems to move simultaniously, with them both stopping automatically when they reach their target position.

A sidebar to this discovery is that anyone who has been doing their own "drive and check encoder" loop probably hasn't seen quite as many driving glitches, because they have been overwriting any partial commands.

With these latest code changes, and some other timeout tweaking, we're feeling a LOT better about our BOT's Auto performance. We did several tests of each of our 8 Auto modes (red/blue, floor/ramp, score/spoil), and never saw a single glitch.

Phew..... Things are looking up

[basicxman]

Originally Posted by emmell

Hmmmm.... I wonder if ROBOTC users see this as well.

NI/PITSCO - please notice this and investigate.

I have done lots of testing with RobotC on two NXT's, I haven't had any problems yet. Although at the competition I had this happen to me, my alliance partners and opposing alliances many many times. This needs to be solved. Hopefully before the championship as it's probably the most backlogged tournament already.

[ptan]

I've barely been able to recreate what actually happened to us at the tournament -- basically at the Ontario FTC Championship, we saw our robot keep moving, even though we had lost all joystick control.

We've been at this for the past week, so this is the first time that we actually saw it happen again.

On the screen of the NXT, it shows that the bluetooth was disconnected (the diamond beside the bluetooth symbol became a less than sign), but the teleop program was still running. The motors were still doing the last thing we told it to do before the bluetooth connection got lost -- AND THE ROBOT DID NOT SHUT DOWN THE MOTORS after the connection was lost.
The NXT screen was still showing "Teleop RUNNING"

I don't know if we can recreate this problem again or not as it was the first time we saw it in the past week of testing and practicing. The FMS showed that the robot was still connected on bluetooth.

This is a brand new NXT (out of an unopened box as we had replaced the original NXT thinking it might have been flaky), Robot C 1.46 with the latest firmware, running the competition templates.

[basicxman]

Originally Posted by ptan

I've barely been able to recreate what actually happened to us at the tournament -- basically at the Ontario FTC Championship, we saw our robot keep moving, even though we had lost all joystick control.

We've been at this for the past week, so this is the first time that we actually saw it happen again.

On the screen of the NXT, it shows that the bluetooth was disconnected (the diamond beside the bluetooth symbol became a less than sign), but the teleop program was still running. The motors were still doing the last thing we told it to do before the bluetooth connection got lost -- AND THE ROBOT DID NOT SHUT DOWN THE MOTORS after the connection was lost.
The NXT screen was still showing "Teleop RUNNING"

I don't know if we can recreate this problem again or not as it was the first time we saw it in the past week of testing and practicing. The FMS showed that the robot was still connected on bluetooth.

This is a brand new NXT (out of an unopened box as we had replaced the original NXT thinking it might have been flaky), Robot C 1.46 with the latest firmware, running the competition templates.

This happened to pretty much every team at least once at the Ontario regional (IIRC). Specifically HiTechnic needs to work on this...

[ptan]

Originally Posted by basicxman

This happened to pretty much every team at least once at the Ontario regional (IIRC). Specifically HiTechnic needs to work on this...

Not sure if this is a HiTechnic problem or not. The NXT was still running the Competition template code when it lost the bluetooth connection -- shouldn't that have stopped the robot by terminating the teleop program? That's what I would have coded for safety (i.e. an out of control robot still running in teleop... think of the burned out motors, let alone the refs running away! -- remember, the NXT will never know when to stop as the bluetooth link disappeared)

[PhilBot]

How is "move a specified distance" implemented in robotc?

In labview, the vi disables the MC's 2.5 sec timeout and then issues a "run to position" command to the MC. LV then sits in a blocked loop waiting for a "done" from the MC. LV uses it's own 10 second timer to abort if the target is not reached.

This strategy has many downsides: the NXT will not respond to a disable, and even if the program is halted, the MC will continue to run until it reaches it's target.

You need a sepparete loop checking enable status to be able to break the motion.

As for the "connected status", this must have a longish timeout. I can turn off my NXT and the status shows Connected until I turn it back on again.

[dcribbs]

We are using NXT-G for our programming and have only had 2 issues. Not sure if #1 is related to memory loss (or whatever it may be) but sounded similar to your experience.

1. We had an extremely long wait from when we connected to the FMS to when the actual match started. Our robot did absolutely nothing. In matches after that, if we had a long wait we would stop our autonomous program and restart it. It seemed to work. We never had that problem again.

2. In the middle of a match one of the controllers stopped working. It was controlling the 4 12v-DC motors. The other controller worked fine controlling our NXT motors and servos.

[chopperdudes]

yeah in the ontario regionals alot of stuff like that happened.

also in one of our matches, we didn't loose COMPLETE joystick controls, but it somehow seemed VERY offset. i'm pretty sure we still had control because i do recall the button we coded for 100% forward still works. but the joysticks seem to be offset, and the robot itself started moving backward and knocked our teammates off the ramp. but with alot of luck we managed to score 2 pucks into the high goal and won the match lol.

the thing is though, i think we wiggled the joystick before the teleop mode, so maybe when it looked for center, it ain't rly center. (i really don't know how to explain this).... so from then on, we wait until at least one of the other robots to move before us moving.

[dickswan]

Regarding the "memory leak" problem.

The LabVIEW / NXT-G firmware can do a lot of dynamic memory allocation that will eventually cause garbage collection (GC) of memory. It may be that GC is not always robust and you're getting the spurious program aborts because of a failed GC command.

Simple blocks like adding two integer variables will not do nemory allocation. But blocks like concatenating a string, display on LCD, etc will. I also think the "get last joystick message from FMS" likely involves a menory allocation.

ROBOTC uses different firmware that is not performing extensive memory allocation.