Driver Station Ghost EStop

While competing at an event over the weekend, our robot had some trouble climbing. It would stop part way up the rope. When looking at the Driver Station logs after the event, we found that a couple seconds after we enabled our climber, the Driver Station said that we used the Driver Station spacebar EStop button. I am fairly sure that the drive team didn’t accidentally press the spacebar 4 times in a row. Any ideas on why we are EStop-ing ourselves?

Could it be PDP breakers overheating instead?

I’m fairly sure it is not the self-resetting breakers. In case you missed what I had wrote, here is the TL;DR: There are EStops via DS spacebar (we didn’t press the spacebar) that are being logged to the DS log. This occurs after we start climbing.

Sounds like the issue that FMSLite has with the E-stop. Only it’s really repeatable to end-of-match sequence…

You don’t have anything sending text of " " to the driver’s station, do you?

FMSlite estop bug was actually only during the very beginning of the match. Similar, but not identical.

Just to extend my description. We had the robot stop both during the match and while tethered in the pits. We only got the log saying that we did the DS spacebar EStop while tethered. We did see the motor output suddenly drop in both cases.

Just realize that with a true eStop, the robot will never move again until the power has been cycled on it.

Try a different laptop for a driver station perhaps? Could be your laptop just has a keyboard that’s on the fritz. As someone who works in IT, this is not an uncommon issue to see, though I suspect in the case of the spacebar it wouldn’t be something you would notice easily. The fact that it only happens during endgame is odd but there could be other contributing factors.

Are you using any custom controls on your Driver station or just normal USB Joysticks and/or game controllers?

As far as I know, there shouldn’t be anything on the robot that would cause an E-Stop, E-stop is initiated strictly from the Driver Station or the FMS.

What we noticed at Palmetto this weekend is that whenever a match ended, the driver station indicated that it had been E-Stopped. When we got the robot back to the pits we had to restart the driver station in order to be able to re-enable the robot again. This did not happen during the match, and our performance was not affected by it, however when we got back to the pits, the driver station always had 4-5 E-Stop indications on it even though we never hit the E-stop or the spacebar.

A few things to check…

  1. Does your driver station randomly start spewing spaces characters sometimes? We’ve experienced this. It takes restarting the laptop to make it stop. It could indicate a dirty keyboard. Clean out the keyboard if possible.

  2. Check the roboRIO itself for metal filings. Sometimes those pernicious little particles get into the worst places causing shorts. You may be able to verify this through the roboRIO lights
    a. comm flashing red is e-stop
    b. power solid red is short
    c. power flashing red is input voltage is too high

  3. If it’s always stopping when you’re climbing, check the voltage in the driver station logs. That shouldn’t have anything to do with e-stop but in the heat of the moment, sometimes it’s easy to overlook. If you are encountering some sort of power issue, you should see it in those logs. Btw, if you are encountering a power problem when climbing and there’s no obvious fault in your electrical, consider that your rope may be getting tangled in such a way that it’s putting much more force on your climber than you anticipated.

Good luck finding the problem!

FYI, we saw a series of “e-stop” messages in our logs while troubleshooting another issue with a CSA at week 1. None of them corresponded to timing of actual problems–that is, the robot didn’t REALLY stop, the logs just said it did. The actual times our bot lost connection were elsewhere in the logs, and we wasted a fair amount of time chasing the e-stop wild geese (red herrings?).

tldr: Check your log timing to be sure that those “phantom e-stops” are really related to your issues.

Oh, and for the record, our “actual problem” turned out to be a wifi jamming sysem at the host high-school that was interfering with the FMS.

Intrusive/immoral school IT strikes yet again

This sort of thing is actually illegal under FCC regulations, but it is all too commonplace. Years back when we first went to WiFi radios we found out our school did the same thing (the school wifi sent out deauthentication packets to disrupt “rouge” wireless APs) and we had to have them disable it to be able to run our robot without it constantly tripping the watchdog.

If you’re interested in the nuts and bolts of it, here are whitepapers on the Cisco Meraki implementation, which we have at our high school:

https://meraki.cisco.com/lib/pdf/meraki_whitepaper_air_marshal.pdf
https://meraki.cisco.com/lib/pdf/meraki_datasheet_airmarshal.pdf

You might want to forward this to your schools network administrators then because the stuff described on page 7 of that first doc is likely considered illegal.

In certain cases it might be allowable to block wireless networks that mimic an existing network or networks that allow wireless connection to an existing network bypassing existing authentication, but my understanding is the FMS system does neither of these things.

All that aside, isn’t WiFi at venues supposed to be disabled per FIRST guidelines anyways?

It’s not realistic to disable as WiFi access points at most venues. Many larger arenas or school buildings have very complicated systems, and disabling those APs is much more complicated than, say, unplugging a router. All we can do is reduce the amount of active wireless networks. That’s why FIRST asks that teams don’t set up hotspots, or run their robots in AP mode.

Our team had this issue during some of our offseason comps and realized that the FRC driver station would be open while they are fixing the code and hitting the spacebar. My suggestion is to close and reopen the driver station console once connected to FMS and that cleared up our issue.

Remember that you can use enter to disable the DS during testing. Doesn’t require a roboRIO restart.

We also had this problem over the weekend. In the pits during testing the driver station would indicate that we had hit the EStop, when we hadn’t. A full reset of the driver station was required for both times this happened.

Then during one of our last matches we were also unable to climb our rope. Our operator happened to be looking at the driver station and saw that our connection was spotty, so I wonder if the two are connected in some way.

Definitely talk to your friendly FTAs, FTAAs, or CSAs. They have access to the FMS logs that can tell you exactly what’s going on with your robot connection, including when the E-stop was pressed.