Our robot is consistently having periods of rapid comms loss, where it rapidly disables and enables, causing our bot to stutter. Here’s a video of driver station having a seizure, and attached the portion of the log file where it happened.
Our radio is powered PoE by the Rev RPM, which has a 10A breaker and is connected to the Rev MPM. It is mounted on a high point of the robot, with a direct line of sight to the laptop, far from motors. Our laptop has all firewalls and security turned off completely, and we are using a 5-port brainbox ethernet switch powered by MPM 10A.
If anyone can help, it would be greatly appreciated, as the robot as it is right now is unusable, and we won’t be able to compete if this issue persists.
In times where we have issues that are unexplainable, I find it’s best to try to subdivide the problem to find where the issue is. First off, I’d suggest commenting out all code in your robot except for empty init/periodic functions, then run that. If you still have this issue, you know it’s not your code. If the robot can sit there happily in this state, then you know the issue is in your code. From there, you can keep going. If the problem isn’t in your code and you suspect your radio, try connecting over Ethernet. Keep using process of elimination to narrow down the source of your problem and then you stand a better chance of solving it.
In the second graph view, the disconnects occur almost exactly every minute. Is the driver station machine a completely clean Windows install with nothing but the DS package?
Don’t have the log on hand to compare, but we and other teams at Esko High School saw that sort of rapid disconnections, and best we could guess it had something to do with the wireless environment (we don’t see it at home, and it hit multiple teams).
Are you using a usb camera directly plugged into the roborio? If so, tried disconnecting it and seeing if you get the same results. We ended up having similar issues, and when we unplug the USB camera and plugged it directly into the limelight it got rid of the issue.
FWIW, hopefully helpful, we had the similar issue at NHGS. Ours was not as significant and did not affect performance too much. We replaced everything (one component at a time). In the end, it turned out to be the ethernet switch (expensive Brainbox). There were no loose connections, it was just the switch itself. Swapped out with cheap switch from Amazon and the problem is resolved.
My team has been having similar troubles throughout this season, and they’ve done some pretty thorough troubleshooting. The issue appears to be isolated between the Driver Station and the radio.
Ethernet or USB tethering into the RoboRIO directly: no problem.
Ethernet tethering into the network switch (connected to co-processor, RoboRIO, and radio): no problem.
Wireless or Ethernet into the radio: communication seizures.
Like others in this thread, we’ve tried a variety of radios, laptops and RoboRIOs, all with the same problem. Radios have been configured as 2.4GHz and 5GHz with still no change. The ethernet cable and POE injector between the radio and switch have both been swapped out.
We tried the 2022 Driver Station software, and did not notice the same issue, though we did notice some intervals of abnormally high packet loss that seemed to align with when we’d receive disconnections in the 2024 software.
I am very confused by this combination of symptoms. For some reason it seems like the 2024 Driver Station has some issue connecting via the radio. If it was just a noisy wireless environment, I’d expect the issue not to happen tethered into the radio. Does the 2024 Driver Station have less tolerance for lost packets before showing “No Robot Communication”? What’s causing those lost packets in the first place?
FWIW the team has only tried OM5P-AC radios. Next up is to try our OM5P-AN to see if there’s any difference.
We had a super similar issue last year. We tried everything we could, but nothing seemed to work (including multiple newer laptops). We eventually went to an older laptop, and the comms issues became more manageable (less than one jitter per match, if at all), though we never really got rid of them. Our coach also said he went in and thoroughly wiped a newer laptop instead of just factory resetting it, and that seemed to fix the issue, but that was months removed from the regular season.
We also have a new issue this year that seems similar to this one described in this thread. They seem to be two different things as one fully disables us while the other only causes a jitter/seizure, but I am not 100% certain.
The timing checks were tightened up for 2024, so yes, I believe this is likely a symptom of that.
There can be a variety of reasons, both hardware and software. Some background updaters (e.g. Inventor) are notorious for causing these kinds of issues. On newer laptops the cause may be power optimizations (either the CPU or WiFi adapter going into a low-power mode) or packet coalescing on the built-in WiFi adapter (a lot of laptops now prioritize power or throughput over latency). Sometimes you can find advanced adapter settings in Windows that can help. Also make sure your laptop is in “high performance” power saving mode.
It may be worth trying to use a separate WiFi router/bridge to connect wirelessly to the robot radio, and connecting the laptop to it through Ethernet. That would at least exclude the WiFi adapter from the equation.
I want to second @Peter_Johnson’s suggestion of using a wireless bridge device instead of the DS laptop’s wireless adapter. Windows and wireless drivers try to provide good behavior for people moving around with their laptops. Networks come and go, some APs are “better” than others; it’s important to conserve battery power; etc. All this means that Windows and drivers are optimized for quite different use case than driving a robot.
Many (not all) home router devices can be configured in “bridge” mode where instead of providing a wireless access point, they instead connect to an existing AP and connect their ethernet ports to that AP’s network. Use of one those to bridge your robot radio’s WiFi network to an ethernet port and locate it close to your driverstation.
When you connect the DS laptop via ethernet to such a device you get rid of all that complicated behavior of Windows and the wireless adapter. The router also probably has way better antennas and more power than the laptop’s wifi adapter.
I experimented with this setup this morning in an environment where I would typically get the rapid disable/enable cycles, flashing red and green on the dashboard, and sawtooth displays in the timing window. Problems gone.
Note that other things can cause this behavior–most of which have been addressed above. But when you have solid power and data connections to all of your devices on the robot and it is still happening, this is something I would try to implement in a heartbeat.
Thanks for the suggestions, @Peter_Johnson and @chauser — configuring a bridge router appears to have worked perfectly, and we’re no longer seeing these disconnections!
