The question came up this weekend in Sacramento as there were at least 3 fast auto can grabbers in the field: Does the FMS enable robots in sequence during auto, or simultaneously? And if its in sequence, is it known which stations enable in sequence and what is the delay between each station?
High speed video of 1678 and 971 in a semifinal match show that a delay could affect match outcome.
Do you have video?
I suggest you view the FMS White Paper to better understand how the field works.
Same thing happened on Dallas week one. There were a couple of can grabbers and 118 showed us high speed footage that showed us not moving for ~60ms from when the opposing alliances robot was moving. We were told that the FMS does enable in a sequence but thatās all we were told. Donāt quote me on the last part.
There is one Ethernet connection between the FMIS Server and the rest of the world. The commands have to go out serially.
Iām guessing the Cisco Wifi router has 3 channels, and 2 robots are assigned to each channel. Wifi Contention could delay the receipt of commands by a robot.
If the command is routed through the Driver Station, then there is minimal delay between when FMIS sends the Auto Start, and when your DS is first able to react.
How fast your robot can react to the DS commands depends upon how much wifi traffic there is on your signal. Also, the Cisco Router has one ethernet connection, so how fast your DS responds will determine where in the que is your command.
This paper does not address how the FMS software goes about enabling robots, only the order in which it traverses match state.
We have not measured exactly delay, but have anecdotal evidence of seeing some robots move before others. Field timing issues were pretty prevalent last year and itās unclear to me any of it has been fixed.
Unfortunately, the delay from the FMS may decide Einstein.
The FMS AP has 6 SSIDs Each robot & DS is own its own VLAN & AP. Presumably there is only one ethernet connection to the FMS server. The FMS server sends the enable to the Driver stations which then send the enable to the robots. I donāt know how you would be able to predict how much jitter is in all that what testing. Since the enable is being processed by the DS, maybe the speed of the DS has some effect as well?
Iād be interested to hear how this was approached with Cheesy Arena.
Based on things Iāve seen in the past, its a mix between ds, fms, fpga and the robot code itself. Based on my understanding I would say the fms actually affects it very little.
I would think that each DS is running its own polling loop asynchronously at ~50Hz (the rate of driver control packets). So even if the disabled bit toggles simultaneously for all six bots from the FMS (which it doesnāt since the FMS->DS packets must be sent serially, but it ought to be close), there could be a 20ms swing in when each DS sends the first enabled packet depending on the phasing.
( I am just guessing at how the FMS and DS are architected here since I donāt think it is publicly documented; please set me straight if I have made an incorrect assumption. )
Realistically, with our wireless links, synchronization is always going to be hard.
It could be multicast to begin to deal with the delay of serially issuing the āchange of stateā messages.
Thereās also another 20+ ms delay caused by the wpilibs if you use anything but the sample robot template. Could end up being up to 40ms depending on how the fpga loops.
Please elaborate?
Hereās a frame by frame of the Dallas Q39 (118 and 3310 in same match).
118 is going for their 3 tote auto, 3310 is going for containers, and a team 932 (middle of field red side) is driving forward at a high speed in auto mode.
Frame 1 - Yellow lights on field illuminate
Frame 2 - Red/Blue DS stack lights illuminate
Frame 3 - Nothing
Frame 4 - 932 tips back and begins movement
Frame 5 - 932 continues to move
Frame 6 - 3310 begins to drop arms
Another frame by frame on Dallas F1 (118 grabbing from step, 148 auto)
Frame 1 - Yellow lights
Frame 2 - stack lights
Frame 3 - Nothing
Frame 4 - 148ās arm moves
Frame 5 - 118 drops arms
From watching a few other matches, there is usually a different number of frames between lights and robot movement for teams like 118 and 3310.
In LV I know that it runs disabled for 1 extra loop after it receives an enabled packet. Then it will run auto on the next enabled packet.
Based on the last time I looked in the Java code, it will run auto init, then the next packet run auto periodic. So unless you are starting your sequence in auto init, it wont run motors until it gets a second enabled packet.
Note that the whole time this is happening, the fpga is enabled, so it will run 20ms with the values it last receives. This is why there are times the robot jumps between disabled and enabled.
I do not think this is currently the case in Java (for IterativeRobot at least). It does check for the new data flag (via nextPeriodReady()), but this flag is not cleared after calling autonomousInit(), so autonomousPeriodic() should be invoked on the first packet
I came here to say the same thing. Regardless of whether the FMS sends out the signals in parallel or in serial, the time between the FMS sending out each packet is going to be negligble compared to any number of other factors in the system. An obvious unknown here is that each teamās driver station has its own limitations due to wildly varying hardware and drivers.
Any anecdotal or video āevidenceā is rather worthless in this discussion without knowing a lot more about each robot (including seeing its code). If team Xās arm appears to move at 0:00:00, but team Yās wheels start spinning at 0:00:20, how do we know which team got the āstart autoā signal first? It is entirely possible that team Yās code is simply slower or explicitly waits 20ms. It is also possible that there are mechanical limitations that caused team Y to start visibly moving later.
It would certainly be interesting to see some hard data on when robots across a number of matches receive the signals, but I wouldnāt rely on any such data being universally accurate, nor would I hope to have perfect synchronicity in robot start times.
Cheesy Arena sends DS packets at match period boundaries (with 10ms resolution) and at 250ms intervals. A single thread of execution decides whether a packet needs to be sent, and sends all six in sequence (by iterating through a hashmap of the driver stations, so the order is not deterministic). The DS packets are UDP, so all six are probably sent over a period spanning far less than a millisecond. I would guess that the six driver stations would receive the āenableā signal within a similar total time frame.
Itās well known that the FMS doesnāt communicate directly with the robot; the FMS informs the DS what state the robot should be in and the DS passes it along with the joystick information. What I donāt know is whether the DS syncs to send a robot packet whenever it receives an FMS packet, or if it only passes on the enable signal next time its regular loop runs. If itās the latter case, then the order and timing of robot enablement is determined presumably by when each driver station process was started and by the DS-to-robot packet interval.
Ideally, the DS should send a packet to the robot as soon as it receives an FMS packet that changes its disable/enable/auto/teleop state, regardless of when it last sent a packet. Someone could probably test whether it does this or not using Wireshark.
Does the command get sent to all 6 DS laptops at the same time? The server in the scorpian case has one ethernet port for all red and one ethernet port for all blue robots. Does the software send red and blue simultaneously, or is there a single network adapter + a switch for both red and blue, meaning one must be sent before another. How much time is between these two commands? Within the separate red/blue disable commands the 1, 2, and 3 station enable commands must happen serially. Are they sent immediately after each other, and if not, what is the delay.
It might seem a silly question to ask if there is a significant delay between sending red and blue side commands, but hey, thatās exactly what happened last year with the hot goal lights, which were +/- 4 seconds in terms of accuracy at times.
These ethernet cables run to control cases (SCCās) under the DS shelf, where they plug into an ethernet switch. This ethernet switch is connected to an Allen Bradley Stratix 8000, which has three other ethernet ports that are dedicated to 1, 2, and 3 driver stations. What goes on here is unknown, but itās not unreasonable to expect that there could be delays here. The Allen Bradley box is doing something other than being a switch because it wonāt work if you plug the wrong team into the wrong ethernet cable.
While itās not explicitly stated, most command systems have a processing loop that will āprocessā incoming commands at a known rate (timed so you donāt chew up your CPU). Now, even if we knew the loop rate we donāt know where in the loop the command will come in; will the command arrive immediately at the beginning of that loop, in the middle, or at the end?
It depends on how the polling is set up. If they check for āFMS says enableā every time they send the command (20 ms), then timing canāt be accurate to more than 20 ms, as you pointed out.
The FMS software and DS are written in C# (or c++, I forget), and LabVIEW, both of which easily support event driven behavior, meaning the driver station may respond to an FMS command as soon as the network buffer reads in the message.
Thatās probably the difference - Iād bet the difference in time it takes FMS to issue the command across all the VLANs to the DSās would be in the nanosecond (microsecond for sure) range.
-MM
Youād think, but the difference in time it took the FMS to send commands to the phillips lighting controller last year was sometimes greater than 2 or 3 seconds. FMS timing seems to be properly understood by nobody.
I think itās fairly widely known that the FMS has been substantially (completely?) rewritten since last season, so anecdotes from 2014 are of limited applicability to 2015.
One of the questions I was taught to ask early on in Grad school is what scale of effect actually matters in a given experiment, and to focus only on those factors that have meaningful impact.
If there are no systematic biases towards particular team stations, how much does a worst case 20ms delay due to polling actually affect a mechanism compared to all the other variables in play?