I just checked the Driver Station logs and we’re getting consistent packet loss of 5% and latency spikes up to over 100 ms. I’ve talked with a few other teams and they’ve got it just as bad if not worse. Any supporting/disagreeing evidence? Is this bad enough that we should make a large case to the FTA?
Talking to your FTA will do more than any Chief Delphi discussion will. That’s what they’re here for, don’t be afraid to ask for help.
If anything, I doubt it is the FMS, and more in line with the AP.
Talk your FTA about your concerns and issues. They want you to succedd as much as you do.
I’d love to know if it was the same field used at 10k lakes. I didn’t look at the logs, but my students reported some latency issues even went to the point where they disabled the camera.
If you decide to talk to the FTA, understand a few things:
- Don’t Delay - the longer you wait, the more they’re going to suspect you’re just whining to get a free replay.
- Problems must be Sustained - Look for sustained high trip times. It’s “normal” for WiFi to have momentary bursts of congestion that cause 2-4 seconds of high latency for robots while the radios are trying to back off and deal with congestion. You’re going to have to show sustained latency over a significant period of time (or a critical period of time, such as the last 20 seconds) in order to get the FTA to consider action for a match.
- Problems must be Systemic - One robot having high trip times isn’t a cause for an FTA to be alarmed. It’s actually normal for one robot whose roboRIO code with abnormally high CPU load, radio in a bad location, Driver Station with misbehaving software, or other factors to cause one robot problems. In these cases, FTAs have generally looked at it as “your problem.” The FTA has tools to look at all of the logs for a match, and can decide if there’s a systemic problem or if it’s “your problem.”
This was relayed to me by an FTA at a Regional Event this year, she had to deal with a LOT of teams who “knew what to say” but the logs wouldn’t back up what they were saying. On top of that, most issues are team problems - “but we didn’t change anything” isn’t always true.
That’s correct. FMS doesn’t cause latency, FMS is the hardware and software you interface with. Latency is a combined factor of your robot radio talking to the Field AP, and the communications between your Driver Station and the Robot through that radio-AP communication link.
It’s also important to understand that WiFi is like a meeting - when one person is talking in a room (channel), nobody else can talk without being disruptive (causing congestion). If someone is talking too slowly, it will cause everyone else to have to wait in order to talk (that’s called “latency”). If a robot radio is inside the robot with motor interference, in a metal box, or under power electronics, the radio will end up having to talk slower to be able to cut through the interference. That impacts all robots in the match, not just the robot with the poorly-positioned radio. So please check your alliance mates, they could be the reason you have latency problems!
-George
This is not right. Or at least I can state this isn’t right based on how I know the radios to be configured though I can’t confirm this because the actual configurations are not general public knowledge.
More specifically, this doesn’t apply for the current generation of technology being used… though, again, I can’t confirm this because the information for the configurations is not made public so for all I can state publicly, FRC robots are stuck using smoke signals.
Radio location on a robot does matter though. Mount them high and ensure the cables are plugged in tight.
Curie match Q4 was replayed due to this. I don’t know if match #17 (IIRC) was replayed for the same reason. Our drivers reported a 2 second lag (probably an exaggeration) which actually resulted in some end-game damage. I’m also told that robots remained in powered motion after T=0. I do know that one team on our alliance was asked to reduce their camera usage following the replay.
I was out there for Q49, which got replayed due to massive lag. From talking to the FTAs, it seems like the curie field (for whatever reason) can’t handle the bandwidth they typically would. We were asked to shut off one of our camera streams to help with the problem. They seemed to have the problem under control after our replay.
Roebling had issues at Houston. I would love to find out if this is the same field.
In short, this is consistent with 2791’s experiences on Curie so far. The FTAs are aware from why I’ve heard and are working hard to manage it as best they can.
This is happening too consistently, to too many different teams, that standard refrains about how it’s probably the teams’ fault aren’t really a gracious thing to dump in here.
Disappointing nonetheless, but what can you do.
I’m curious if the original poster can post his drivers logs. It may help diagnose if it is a robot or field problem in his specific case.
Same, why not.
We’ve found that some of the problems with high packet loss are due to our motors consuming an insane amount of power when we try to switch direction or stop (enough to brown out the VRM if we’re pressurizing at the same time), and some of it is also probably us having an older radio. Thanks so much to the FTAs and Jacob for helping us understand and fix our problems, and thanks to you guys for letting me know that I shouldn’t be worried about my request wasting their time!
Thank you for coming back here and letting us know what the real problem was.