Feedback Thread: Robot Control System

[Greg McKaskle]

If someone has anything, I'd like additional symptoms or triggers on the inability to connect to the FMS. The code reads UDP from the port with a one second timeout, processes anything it received, and loops around for more.

Any idea if a laptop that fails to connect has ever connected? Is it the laptop settings or other issues?

Any idea if it tends to happen after a suspend, a reboot, or ???

To try and reproduce this, was this FMS lite or the real deal?

Greg McKaskle

[Mark McLeod]

Here's what I know...
I worked on about a dozen of these particular FMS-fail to connect cases at official events with the full FMS. That's why I'd like to hear more reports from other areas of the FRC world to see what patterns others saw.

Under competition circumstances we're just trying to get them operating again, so we don't get much opportunity to study the problem. We just try to make it go away as fast as possible. I never had the time to try it against FMS-lite. I meant to do so in Atlanta, but my laptops were all tied up when the opportunity arose.

• It seemed to manifest when the Classmate woke from sleep, but only a small percentage of Classmates suffered.
• Once it started, rebooting did not improve the problem.
• It can be fixed by restoring the Classmate to KOP as-delivered (~2 hours).
• It can be fixed by a bizarre use of the restore stick (~2 minutes).

I can't say that restoring prevented the problem from coming back. Standard advice quickly became "don't ever let it go to sleep."
When some of them did go to sleep again the problem did return, but that's a very subjective observation and I wouldn't put any faith in it without collecting less-subjective samples.

There were a lot of cases at the week 1 NJ Regional, but I didn't see a single one at the 4th week SBPLI Regional. One big difference is that most teams at SBPLI had them on constant power or shut them down completely while waiting in the queue. By week 4 the sleep problems had been widely reported and teams weren't letting them sleep.

You saw one of these in Atlanta with team 192 in the Curie pits, I think it was.
Taking 192 as an example, they reportedly worked fine with FMS at their regional, but failed to connect to FMS during the Championship Thursday practices. It was fixed at the field and tested fine with FMS. We rebooted their Classmate from scratch and it still connected fine.
Unfortunately, they did not have an inverter to keep the Classmate powered and instead put it to sleep for the long walk and wait in the queue. When it woke up it no longer recognized FMS.

What interests me most about Andy's report is the attribution to conflicts between the Developer and Driver user accounts. I often had teams running both accounts so some of the sleep problems could be quickly fixed. The only FMS problem with having both accounts open that I knew about was early on when users could accidentally run two copies of the Driver Station software, but you guys prevented that with one of your updates.

[Greg McKaskle]

Thanks for the info. Anyone else with observations?

Greg McKaskle

[Radical Pi]

I have never seen this issue on our DS. We were at 2 regionals (FLR and philly). At both regionals I was always sleeping the classmate before each match to save battery. At FLR we had the cypress plugged into the usb hub along with 2 other joysticks (KOP I believe), the hub being plugged in with only the black cable. For philly, the cypress was on the 2nd classmate port and it was being removed before each sleep. In software, the only changes from the default setup were using a custom dashboard, but no data was being sent by the robot (was used for the camera only, no tracking data). I had also bypassed the lockout on task manager in Driver mode for debugging early in the season and never bothered to re-enable it. We did lose connection once or twice on the field, but I believe that was related to our broken ethernet port, since it never happened again after we duct-taped an extension cable to our port.

At FLR though, our two alliance partners in quarterfinals could not connect to the field, but seemed to be tethering fine. One of them got a new classmate after the first match and was able to play in the second. I'm not 100% sure this is true, but I think I saw the default dashboard on both of their robots, and both were using the cypress, and I think they may have slept too.

Has anyone checked to see what happens if a classmate is waked up when the ethernet cord is plugged in? I was having our drivers make sure the classmate was fully running before plugging into the field.

On another note, would it be possible for NI to release an optional update to the DS that includes the cypress fix so that it can be gone for the remaining offseason events?

[Greg McKaskle]

On another note, would it be possible ...

To recap the conversation about the Cypress issue, I was personally able to provoke what looked like an issue with the Cypress device driver by sleeping the laptop. It didn't happen each time, but in ten sleep/awake cycles, I saw it once and figured that was good enough to write a bug report. Yes, unfortunately this test was run after the season was already underway. My preference would be to let FIRST determine how I/O will be done for the DS in upcoming years, and include sleep/awaken tests to verify it works as intended and document any workarounds or configuration issues that will improve the issue.

If the same HW is to be used and it is not possible to get the sleep issue fixed at the driver level, it should be possible to update the DS to restart the driver when it appears to be hung. Code to diagnose and restart the driver has not been written, or I'd gladly let encourage FIRST to release it. In general, I'd prefer to fix the root problem instead of bonking other SW over the head when they don't seem to be responding.

Greg McKaskle

[Tom Line]

I may reiterate what others have already said, but I want to be thorough:

I. Classmate
A. Watchdog problems are still an issue. We fixed the majority by changing powersave and bios options, but it would be nice to see this as part of the formal instructions for rookies AND veteran teams.
B. Transition times are bad between the driving account and the developer. Even worse is the boot time, and the time required to log back out then back in after being FMS locked.
C. When IN the driver station mode, the USB can be problematic. We've had multiple instances this year of it refusing to acknowledge the logitech gamepads we are using.
D. The USB is not consistent in determining the joystick order. It had a bad habit of swapping the way they are installed (when they do install).
E. The E-stop wait. Dump it. A warning sign is enough.
F. PLEASE allow us to use any laptop instead of the classmate. I can buy a $200 ebay laptop that costs less than the classmate, has a longer battery life and more processor power, can be used to program AND as the driver station, and a whole host of other things as well. We used a normal laptop during most of the beta test and never had an issue.

II. PD board.
A. It would be nice if could change to standard-size nuts rather than metric on the positve and negative leads.
B. It would be nice if the wagos could be turned to point upward, so that it isn't nearly impossible to work with them once the PD board is installed on the robot.

III. Crio
A. Power port does not have a positive enough connection to the wires.
B. A short ethernet cable and female-female connector should be mandatory as part of the robot wiring. This is what our team has been doing without issue to prevent the over-use of the crio ethernet port.

IV. Can
A. It would be nice if a can-manual or a "Can for Dummies" came along with next years instructions. This year, unless you're fairly electrically savvy, most teams shied away from Can. It opens some great possibilities....

V. The oft-maligned game adapter.
A. We hot-glued the reset button and the power button. Our robot still reset once from an extremely hard hit. Would it really be that hard to take a tiny form-factor bridge and mount it inside a cradle that fits in the crio itself? Then it could pull power from the Crio, be easily removable, etc.

B. The backup game adapter is horrid. Connection times are ridiculous. We bought one as a backup, but never used it except on the practice bot because of the start-up time. It would have made it a crunch to even set our autonomous mode on the field.

VI. Joysticks
A. I think it would be smart to substitute the commonly used logitech gamepads for the attack joysticks. The gamepads are simply more flexible (requiring only one pad for tank style drive) and are easier to transport. They also allow your drive team more flexibility in where they stand and how they move around.

[biojae]

Originally Posted by Tom Line

VI. Joysticks
A. I think it would be smart to substitute the commonly used logitech gamepads for the attack joysticks. The gamepads are simply more flexible (requiring only one pad for tank style drive) and are easier to transport. They also allow your drive team more flexibility in where they stand and how they move around.

Even though the joysticks come in the KOP does not mean that they have to be used. The gamepads are cheap enough that if any team wants to use them, then they can (about $20).

[fireyoshi]

Our team had a problem connecting to FMS during Bayou. We troubleshooted, and we found out that whenever we tethered the robot, the subnet mask would always reset to 255.0.0.0. I had to manually go the tcp/ip settings before every match and change the subnet back to 255.255.255.0. This fixed everything.

We also had the all-too-common loose ethernet port problem.

During this match at the 1:14 mark you can see 3389 ramming into the alliance station wall causing the ethernet cord to fall out. I plugged it back in fast enough for it to reconnect.