Chief Delphi

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)
-   General Forum (http://www.chiefdelphi.com/forums/forumdisplay.php?f=16)
-   -   FMS enabling sequence? (http://www.chiefdelphi.com/forums/showthread.php?t=136034)

Jared Russell 23-03-2015 16:59

Re: FMS enabling sequence?
 
Quote:

Originally Posted by Thad House (Post 1461209)
Based on the last time I looked in the Java code, it will run auto init, then the next packet run auto periodic. So unless you are starting your sequence in auto init, it wont run motors until it gets a second enabled packet.

I do not think this is currently the case in Java (for IterativeRobot at least). It does check for the new data flag (via nextPeriodReady()), but this flag is not cleared after calling autonomousInit(), so autonomousPeriodic() should be invoked on the first packet

Bryan Herbst 23-03-2015 17:12

Re: FMS enabling sequence?
 
Quote:

Originally Posted by Jared Russell (Post 1461188)
I would think that each DS is running its own polling loop asynchronously at ~50Hz (the rate of driver control packets). So even if the disabled bit toggles simultaneously for all six bots from the FMS (which it doesn't since the FMS->DS packets must be sent serially, but it ought to be close), there could be a 20ms swing in when each DS sends the first enabled packet depending on the phasing.

( I am just guessing at how the FMS and DS are architected here since I don't think it is publicly documented; please set me straight if I have made an incorrect assumption. )

Realistically, with our wireless links, synchronization is always going to be hard.

I came here to say the same thing. Regardless of whether the FMS sends out the signals in parallel or in serial, the time between the FMS sending out each packet is going to be negligble compared to any number of other factors in the system. An obvious unknown here is that each team's driver station has its own limitations due to wildly varying hardware and drivers.

Any anecdotal or video "evidence" is rather worthless in this discussion without knowing a lot more about each robot (including seeing its code). If team X's arm appears to move at 0:00:00, but team Y's wheels start spinning at 0:00:20, how do we know which team got the "start auto" signal first? It is entirely possible that team Y's code is simply slower or explicitly waits 20ms. It is also possible that there are mechanical limitations that caused team Y to start visibly moving later.


It would certainly be interesting to see some hard data on when robots across a number of matches receive the signals, but I wouldn't rely on any such data being universally accurate, nor would I hope to have perfect synchronicity in robot start times.

Pat Fairbank 23-03-2015 17:13

Re: FMS enabling sequence?
 
Quote:

Originally Posted by dcarr (Post 1461184)
I'd be interested to hear how this was approached with Cheesy Arena.

Cheesy Arena sends DS packets at match period boundaries (with 10ms resolution) and at 250ms intervals. A single thread of execution decides whether a packet needs to be sent, and sends all six in sequence (by iterating through a hashmap of the driver stations, so the order is not deterministic). The DS packets are UDP, so all six are probably sent over a period spanning far less than a millisecond. I would guess that the six driver stations would receive the "enable" signal within a similar total time frame.

It's well known that the FMS doesn't communicate directly with the robot; the FMS informs the DS what state the robot should be in and the DS passes it along with the joystick information. What I don't know is whether the DS syncs to send a robot packet whenever it receives an FMS packet, or if it only passes on the enable signal next time its regular loop runs. If it's the latter case, then the order and timing of robot enablement is determined presumably by when each driver station process was started and by the DS-to-robot packet interval.

Ideally, the DS should send a packet to the robot as soon as it receives an FMS packet that changes its disable/enable/auto/teleop state, regardless of when it last sent a packet. Someone could probably test whether it does this or not using Wireshark.

Jared 23-03-2015 17:16

Re: FMS enabling sequence?
 
Quote:

Originally Posted by MasterMentor (Post 1461211)
On the contrary, I think I figured it out directly from the FMS Whitepaper.

From the FAQ, "Does the FMS Control the Robot", FMS sends the commands to the DS, which then forwards the command to your robot to start.

Does the command get sent to all 6 DS laptops at the same time? The server in the scorpian case has one ethernet port for all red and one ethernet port for all blue robots. Does the software send red and blue simultaneously, or is there a single network adapter + a switch for both red and blue, meaning one must be sent before another. How much time is between these two commands? Within the separate red/blue disable commands the 1, 2, and 3 station enable commands must happen serially. Are they sent immediately after each other, and if not, what is the delay.

It might seem a silly question to ask if there is a significant delay between sending red and blue side commands, but hey, that's exactly what happened last year with the hot goal lights, which were +/- 4 seconds in terms of accuracy at times.

These ethernet cables run to control cases (SCC's) under the DS shelf, where they plug into an ethernet switch. This ethernet switch is connected to an Allen Bradley Stratix 8000, which has three other ethernet ports that are dedicated to 1, 2, and 3 driver stations. What goes on here is unknown, but it's not unreasonable to expect that there could be delays here. The Allen Bradley box is doing something other than being a switch because it won't work if you plug the wrong team into the wrong ethernet cable.


Quote:

While it's not explicitly stated, most command systems have a processing loop that will "process" incoming commands at a known rate (timed so you don't chew up your CPU). Now, even if we knew the loop rate we don't know where in the loop the command will come in; will the command arrive immediately at the beginning of that loop, in the middle, or at the end?
It depends on how the polling is set up. If they check for 'FMS says enable' every time they send the command (20 ms), then timing can't be accurate to more than 20 ms, as you pointed out.

The FMS software and DS are written in C# (or c++, I forget), and LabVIEW, both of which easily support event driven behavior, meaning the driver station may respond to an FMS command as soon as the network buffer reads in the message.

Quote:

That's probably the difference - I'd bet the difference in time it takes FMS to issue the command across all the VLANs to the DS's would be in the nanosecond (microsecond for sure) range.

-MM
You'd think, but the difference in time it took the FMS to send commands to the phillips lighting controller last year was sometimes greater than 2 or 3 seconds. FMS timing seems to be properly understood by nobody.

MikeE 23-03-2015 17:44

Re: FMS enabling sequence?
 
Quote:

Originally Posted by Tom Bottiglieri (Post 1461164)
Field timing issues were pretty prevalent last year and it's unclear to me any of it has been fixed.

Quote:

Originally Posted by Jared (Post 1461223)
It might seem a silly question to ask if there is a significant delay between sending red and blue side commands, but hey, that's exactly what happened last year with the hot goal lights, which were +/- 4 seconds in terms of accuracy at times.

Quote:

Originally Posted by Jared (Post 1461223)
You'd think, but the difference in time it took the FMS to send commands to the phillips lighting controller last year was sometimes greater than 2 or 3 seconds. FMS timing seems to be properly understood by nobody.

I think it's fairly widely known that the FMS has been substantially (completely?) rewritten since last season, so anecdotes from 2014 are of limited applicability to 2015.

One of the questions I was taught to ask early on in Grad school is what scale of effect actually matters in a given experiment, and to focus only on those factors that have meaningful impact.
If there are no systematic biases towards particular team stations, how much does a worst case 20ms delay due to polling actually affect a mechanism compared to all the other variables in play?

Tom Bottiglieri 23-03-2015 17:52

Re: FMS enabling sequence?
 
Quote:

Originally Posted by MikeE (Post 1461235)
If there are no systematic biases towards particular team stations

That is the part I am interested in learning more about.

plnyyanks 23-03-2015 18:11

Re: FMS enabling sequence?
 
On a different note, I would love an update to the FMS Whitepaper that goes into much greater technical detail. Or maybe a 1 day recorded conference, where someone on FRC engineering can do a very detailed, technical walkthrough of how the field works (and maybe show some code snippets), including a Q&A.

I don't like how the field electronics are a 'black box' to most teams - I gained most of my knowledge about how the field works from volunteering and working with it regularly, something teams don't do.

A greater understanding about how the competition field works will reduce the amount of anger and frustration often (mis)directed at FMS.

YPatrick 23-03-2015 19:27

Re: FMS enabling sequence?
 
Slow-Mo: https://docs.google.com/file/d/0B9RP...U2dXVkSEU/edit

Real Time: https://docs.google.com/file/d/0B9RP...hIWlotdEE/edit

There is an interpolated clip that is a little bit slower, so if people are interested I could post the link to that one as well.

Citrus Dad 23-03-2015 20:51

Re: FMS enabling sequence?
 
Quote:

Originally Posted by Pat Fairbank (Post 1461221)
Cheesy Arena sends DS packets at match period boundaries (with 10ms resolution) and at 250ms intervals. A single thread of execution decides whether a packet needs to be sent, and sends all six in sequence (by iterating through a hashmap of the driver stations, so the order is not deterministic). The DS packets are UDP, so all six are probably sent over a period spanning far less than a millisecond. I would guess that the six driver stations would receive the "enable" signal within a similar total time frame.

It's well known that the FMS doesn't communicate directly with the robot; the FMS informs the DS what state the robot should be in and the DS passes it along with the joystick information. What I don't know is whether the DS syncs to send a robot packet whenever it receives an FMS packet, or if it only passes on the enable signal next time its regular loop runs. If it's the latter case, then the order and timing of robot enablement is determined presumably by when each driver station process was started and by the DS-to-robot packet interval.

Ideally, the DS should send a packet to the robot as soon as it receives an FMS packet that changes its disable/enable/auto/teleop state, regardless of when it last sent a packet. Someone could probably test whether it does this or not using Wireshark.

That's interesting. So an alliance might want to be careful as to the order in which it connects drivers stations. (And this can be a real world issue.)

Thad House 23-03-2015 20:53

Re: FMS enabling sequence?
 
Quote:

Originally Posted by Jared Russell (Post 1461214)
I do not think this is currently the case in Java (for IterativeRobot at least). It does check for the new data flag (via nextPeriodReady()), but this flag is not cleared after calling autonomousInit(), so autonomousPeriodic() should be invoked on the first packet

Ah. They might have changed that then.

However, LV still does do this, and runs disabled one last time, and still has to wait for another packet to be sent before actually enabling auto. so auto in LV will have a delay of one packet before actually running auto code.

Citrus Dad 23-03-2015 20:55

Re: FMS enabling sequence?
 
Quote:

Originally Posted by MikeE (Post 1461235)
If there are no systematic biases towards particular team stations, how much does a worst case 20ms delay due to polling actually affect a mechanism compared to all the other variables in play?

If the serial enabling differences are cumulative, the first bot will move 0.1 second before the last one. That's more than the difference in speed between several of the can grabbers at the moment.

Thad House 23-03-2015 21:02

Re: FMS enabling sequence?
 
Quote:

Originally Posted by Citrus Dad (Post 1461314)
If the serial enabling differences are cumulative, the first bot will move 0.1 second before the last one. That's more than the difference in speed between several of the can grabbers at the moment.

It would never be that high. Even serially, I bet it still sends all the enable packets within 1ms of each other. The 20ms polling loops happen on the DS, and the FMS does not depend on it. So the FMS wouldnt accumulate based on those. It would only matter to the DS at which point in the 20ms period it recieves the enable packet, and thats if its not interrupt based. So worst case is that 1 robot gets an enabled packet 19ms after another one.

Now its entirely possible something else in code is delaying more, but I cant see how the FMS would take 60+ms to send enable packets. If so, we would visually see it.

FrankJ 24-03-2015 09:58

Re: FMS enabling sequence?
 
Quote:

Originally Posted by Jared (Post 1461223)
...
These ethernet cables run to control cases (SCC's) under the DS shelf, where they plug into an ethernet switch. This ethernet switch is connected to an Allen Bradley Stratix 8000, which has three other ethernet ports that are dedicated to 1, 2, and 3 driver stations. What goes on here is unknown, but it's not unreasonable to expect that there could be delays here. The Allen Bradley box is doing something other than being a switch because it won't work if you plug the wrong team into the wrong ethernet cable.
...
FMS timing seems to be properly understood by nobody.

The switches are managed. Which means they only route packets to specific ports based on the IP address (or MAC). Managed switches are always higher end than generic switches. You are not losing any significant time in the switches. The travel time for Ethernet packets between the DS & robot are logged on the DS. I expect the travel times between FMS & the DS are the same or better.

I expect the FMS timing is understood by the people involved in writing the programs. It just not public information.

themccannman 08-04-2015 02:28

Re: FMS enabling sequence?
 
So, since this doesn't seem resolved is there any way we can figure this out aside from FIRST publicly posting the FMS code? They only other way I can think is setting up two robots with known mechanism speeds and comparing them at each possible driver station combination. This would be pretty difficult to do though as you would need to reserve an entire field for only 2 teams for at least 30-60 mins, likely not possible with time constraints and demand for practice matches at most competitions.

MrRoboSteve 08-04-2015 10:33

Re: FMS enabling sequence?
 
I have no knowledge of the internals of the FMS software, so these observations are based completely on the whitepaper and other public sources.

1. The minimum interpacket gap for 100mbps Ethernet is 0.96 µs.

2. The FMS code could take longer than 0.96 µs to send the "auto enable" command to each DS, depending on what the logic looks like in the FMS code. For example, there could be logging being written to disk between each initiate command.

3. The protocol between the FMS and the DS isn't documented in the whitepaper.

4. The FMS whitepaper says that robot <--> DS control packets are prioritized over video packets. I read that as saying that UDP 1130 and 1140 are prioritized over the other ports available for robot <--> DS communication. The whitepaper is silent about the relative priority of FMS <--> DS packets. A reasonable guess is that FMS <--> DS packets are at the same as robot <--> DS control packets. If this guess is true, it's unlikely that robot -> DS data on the open ports (e.g. camera packets) from the robot will interfere with the "auto enable" command from the FMS.

5. It is likely that one of these two statements is true:
a. The "auto enable" command is sent to each DS in the same order in every match (e.g., Red 1/2/3, Blue 1/2/3)
b. The "auto enable" command is sent to each DS in a purposely randomized order in every match

Why are these likely? Because a) is the easiest to code, and b) would be the logical alternative if they made a specific design decision here.

6. The FMS developers are likely to be curious about a thread on CD entitled "FMS enabling sequence."


All times are GMT -5. The time now is 06:55.

Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi