View Full Version : 2015 Between-Match Delays
Qbot2640
03-04-2015, 10:25
I may wrong, but it seems to me that the delays between matches can be particularly long this year. Before the season started, I anticipated field reset to be particularly difficult with the large number of game pieces and the mess they can create, but most events I've seen have handled this extremely well. However, there still seem to be some extremely long delays while everyone is just waiting for the match to start. Is my perception correct?
And if so...is there a reason for this?
I'm starting to give some thought to the schedule for our fall off-season, and just want to make sure I take everything I can possibly predict into account.
BrendanB
03-04-2015, 10:28
Transportation configuration is a big part of the delays. Some teams need to dis-assemble parts of their robot before they can remove their robot from the field and some robots in the next batch of teams need to get their robot ready for the start of the match.
Add in the usual field delays of getting robots connected and it gets hard to stick to the match schedule for time.
I've experienced three or four delays on the field from our team so far, all of which were caused by issues with mDNS.
The laptop will say "FMS Connected, correct station", and our robot will show up as connected with the battery voltage and everything, but the field people cannot see that our laptop is connected. We must disable/enable the network adapter, restart the driver station software, and sometimes even restart the robot.
I've also noticed similar things happening to our alliance partners, where their laptops don't connect to the field, and they have to restart things or try connecting and disconnecting again and again to connect to the field.
The field seems to always connect to the robots in the end, but this year, it seems to take longer on average for all teams to get full communications.
I've experienced three or four delays on the field from our team so far, all of which were caused by issues with mDNS.
The laptop will say "FMS Connected, correct station", and our robot will show up as connected with the battery voltage and everything, but the field people cannot see that our laptop is connected. We must disable/enable the network adapter, restart the driver station software, and sometimes even restart the robot.
I've also noticed similar things happening to our alliance partners, where their laptops don't connect to the field, and they have to restart things or try connecting and disconnecting again and again to connect to the field.
The field seems to always connect to the robots in the end, but this year, it seems to take longer on average for all teams to get full communications.
Yes, at Bayou we had a number of matches delayed for FMS connect issues, two our team, three or four others. I did not know whether they were robot or driver station end.
Conor Ryan
03-04-2015, 10:51
Matches this year are generally 8 minutes in length (End to End, including the actual match). Most of the delays are due to the set up times required for the whole transportation configuration stuff. Field and connectivity issues are rare, but do occur.
Generally after everyone has been on the field once, schedule a 10 minute break, you'll use the time for sure and after that the schedule tends to stay true. Getting that first match out of the way is difficult, where to go, who does what and various other nonsense.
Other than that build 5 minutes for later in the morning issues and 5 minutes in the early afternoon and you'll be on schedule if you get lucky.
ayeckley
03-04-2015, 11:32
Field and connectivity issues are rare, but do occur.
While this was generally true at Pittsburgh it was not the case at Buckeye. I'd estimate (based on my personal observation) that on Thursday less than 25% of Practice Matches required FTA/FTAA involvement including going out to the Alliance Stations. That rate shot up to over 80% for Qualification Matches on Friday, and back below 25% on Saturday.
Anyone know which field is at Windsor?. I'm tempted to say the field at Buckeye was #11 but there a >50% chance that I'm mistaken.
Jared Russell
03-04-2015, 11:45
mDNS can make establishing the connection to the robot take for-freaking-ever. If anything, the large number of game pieces and long robot setup times help to mask just how long the delay can be.
MrTechCenter
03-04-2015, 12:24
We ran into a similar issue at Central Valley in Week 2. We were behind schedule quite a bit, so we had teams unplug their driver stations immediately after the match ended and the next match would plug their stations in as quickly as possible. The driver stations for the next match would connect to the FMS, but the next match wouldn't be pre-started yet, so they wouldn't see communications with their robot yet. Most of the time, we had to have teams restart the driver station application. But even then, the delay wasn't huge.
I think the biggest delays come during eliminations, because alliances will face back-to-back situations and this requires a field timeout. Tack on a timeout or two and there's a slightly longer today. Still, most regionals are ending on time or earlier than expected.
mDNS can make establishing the connection to the robot take for-freaking-ever. If anything, the large number of game pieces and long robot setup times help to mask just how long the delay can be.
Once a good rhythm is established the various sources of delay tend to align, i.e. they all mask each other and it appears smooth and painless on the surface.
At a couple of my events we timed the inter-match delay and it was interesting to see how much perception and reality were uncorrelated.
...
Anyone know which field is at Windsor?. I'm tempted to say the field at Buckeye was #11 but there a >50% chance that I'm mistaken.
I'm at Windsor. How do you tell what field is here?
ayeckley
03-04-2015, 14:26
I'm at Windsor. How do you tell what field is here?
The road cases should have the number printed on them. I don't recall where they were stored last year; I think in the north-west corner of the gym near the mentor lounge.
The road cases should have the number printed on them. I don't recall where they were stored last year; I think in the north-west corner of the gym near the mentor lounge.
I'll check tomorrow. One of the volunteers said that the US kits stay in the US and the Canada kits stay in Canada. Either way, I'll try and verify tomorrow.
Usually at Bayou, by the time the score had been settled, the robots cleared and replaced, and the field reset (which usually went pretty briskly considering that there are over 200 game pieces), the match was able to start within another minute or so; Chris Copleand often started announcing matchups before the last team was off the field to help achieve this.
About one match in four or five seemed to have extra delays. Of the ones where I could see or later heard the reason, two were improper robot placement (hanging over the landfill or auto zone) and the rest (about five or six) were connectivity issues (one robot, the rest driver station). I suspect most of the others were connectivity, as well; I could occasionally see one team working hard behind the polycarbonate while the others waited.
Greg McKaskle
03-04-2015, 19:35
The "mDNS" issues are something that is still being investigated. The better name for it might be "slow DS-robot connection on field", because there are a few glitches here and there that show similar symptoms, but different reasons for the delay/breakage. Also, the necessary DS restarts are caused by a few error conditions that require the TCP comms between FMS and DS to restart. They weren't discovered during beta and show up more often on some laptops than on others.
Good news is that some fixes are being rolled out with field configuration. Others would require new DS or library releases, but there is considerable overhead with a mid-season release. So I guess the summary is that your perception is correct, and it is something that is being worked on more than it may appear.
Greg McKaskle
I'll check tomorrow. One of the volunteers said that the US kits stay in the US and the Canada kits stay in Canada. Either way, I'll try and verify tomorrow.
Don't rely on the road case number: 11 is the ref case, 19 is spare FMS console, 07 is field supplies IIRC. The small tags might or might not have anything regarding which field.
You want the case COLOR. All the cases in one field match in color--and one or more of them are generally kept right next to the field. Example: Gold field is currently in SVR, after trekking to Inland Empire, CVR, Los Angeles, Sacramento, and Ventura. (Though, I think Gold field is #6, at least it uses radio set #6.)
plnyyanks
03-04-2015, 21:12
The laptop will say "FMS Connected, correct station", and our robot will show up as connected with the battery voltage and everything, but the field people cannot see that our laptop is connected. We must disable/enable the network adapter, restart the driver station software, and sometimes even restart the robot.
This typically happens when teams are plugged in before the field has completed prestart (and is usually worse on practice day, when we're making substitutions. From being fieldside at two events now, I've found that restarting the driver station or changing the team number almost always solves the problem (when the field monitor shows a ethernet link and nothing else, or everything but roborio). If not, restarting the NI mDNS responder service does the rest. I'm seeing this at Chesapeake, and it's consistent with what I saw at NYC (http://www.chiefdelphi.com/forums/showthread.php?p=1457911#post1457911).
I think the faster turnaround time from the refs as they hit their groove (so the field can complete prestart before the next round of teams plugs in) is the biggest factor in these delays not being seen later in the competition.
Anyone know which field is at Windsor?
I believe the Windsor field was previously at Montreal and North Bay.
(Though, I think Gold field is #6, at least it uses radio set #6.)
The radio set and truck number are two different things. Truck #2 (lime green cases) has radio set #4.
Matches got behind schedule at Northern Lights from 20 minutes to ~40 and back down to <20; (ie a team supposed to be on the field at 12:00 didn't play until anywhere from 12:20 to 12:40), delays were sometimes really long. (Noodle throwing contest anyone?)
North Star, we've stayed about 20 minutes behind. Some matches take as little as resetting the field and the like, some took 10 minutes or more.
I'd say allowing an entire match cycle at 8 minutes as noted before is good timing.
orangemoore
03-04-2015, 22:57
At Midwest we have an aggressive 7 minute pace for match times. We were actually on schedule for most of the day and I think at one point we were ahead of schedule.
(I was actually surprised that each team was only given 10 matches. Usually it seems that Midwest pushes the schedule to the max.)
Maybe it helps that there are no rookie teams?
GaryVoshol
04-04-2015, 06:27
At the Michigan events I've volunteered at, the time consumers are, in order (from least to most time):
Finalizing and announcing scores
Field Reset
Robots on and off the field
Getting robots connected
Yesterday afternoon we stressed that moving robots was more important than moving totes. This improved the timing of getting the robots onto the field, so they could begin getting out of transport config faster.
When things run well, you can keep to a 7-minute cycle, with occasional 6's or 8's. But then you have a big delay where some robot just will not connect, and the FTA's and Orange Hats are working feverishly. Sometimes it takes 2 or 3 dance songs before you're ready to go, and this year there's no way to make up that time.
I didn't count, but in a normal 80-match qualification in FiM, there's probably 2 or 3 times that the green light goes on before the announcer is done introducing the teams. There's maybe 5 or 6 matches where we're waiting for last minute adjustments to the robot after the light is green, and that only for a few seconds. The rest of the time everything is ready to go except the electronics.
Ginger Power
04-04-2015, 08:28
I've experienced three or four delays on the field from our team so far, all of which were caused by issues with mDNS.
The laptop will say "FMS Connected, correct station", and our robot will show up as connected with the battery voltage and everything, but the field people cannot see that our laptop is connected. We must disable/enable the network adapter, restart the driver station software, and sometimes even restart the robot.
I've also noticed similar things happening to our alliance partners, where their laptops don't connect to the field, and they have to restart things or try connecting and disconnecting again and again to connect to the field.
The field seems to always connect to the robots in the end, but this year, it seems to take longer on average for all teams to get full communications.
We have had this problem as well, but to a greater degree. So far we have only been fully connected to the field without issues for one match (and during that match our alliance partner ran into our driver station wall and our robot somehow disconnected again...). We had a practice robot which ran with very minimal problems for 20+ hours. The transfer of parts over was smooth and yet somehow we have stacked 1 tote in 7 matches because our robot has been rapidly disconnecting and reconnecting. With 20+ hours of driver experience I was sure hoping for a better showing than that. For reference we had 5 CSA's in our pit for over an hour on Thursday night and the problem was never resolved. I'm a little frustrated but hope my team can turn it around for our final 3 matches today so all of our hard work wasn't for almost nothing for no competiton results. I'll most likely do a long writeup about my experience with Recycle Rush after today.
MrForbes
04-04-2015, 09:22
Some of the four regionals that I've attended that didn't have Kate as FTA seemed to have many long delays between matches. Here at AZ West, she's keeping things on time and running smooth.
It's an interesting phenomenon.
Don't rely on the road case number: 11 is the ref case, 19 is spare FMS console, 07 is field supplies IIRC. The small tags might or might not have anything regarding which field.
You want the case COLOR. All the cases in one field match in color--and one or more of them are generally kept right next to the field. Example: Gold field is currently in SVR, after trekking to Inland Empire, CVR, Los Angeles, Sacramento, and Ventura. (Though, I think Gold field is #6, at least it uses radio set #6.)
I wasn't able to see any case numbers. Like you said, I was able to see the different numbers on the crate. I _think_ they were red crates, but now I'm not really sure. Doh!
Anthony Galea
06-04-2015, 08:08
I wasn't able to see any case numbers. Like you said, I was able to see the different numbers on the crate. I _think_ they were red crates, but now I'm not really sure. Doh!
It was red cases, and if the number really matters the one I saw said 07.
plnyyanks
06-04-2015, 08:30
It was red cases, and if the number really matters the one I saw said 07.
Every field has a set of cases numbered the same, but with different colors to differentiate them. Case 7 contains the scoring table electronics.
MrForbes
06-04-2015, 08:31
The "red case" field was working pretty well for us at AZ West.
Qbot2640
06-04-2015, 08:53
So back to my original concern...do you think I will be safe planning for 8 minute turnaround...as long as we can keep field reset, and robot-transport configuration-transition within reasonable times as well? I feel comfortable predicting physical setups...just have no idea what to expect with the networking resets.
I am assuming that by October, more of these quirky connection issues will be figured out and solved. I want to figure out how "big" of an event we can plan without running too long or sacrificing number of games played per team.
Alan Anderson
06-04-2015, 10:36
I am assuming that by October, more of these quirky connection issues will be figured out and solved.
There is one systemic problem that can cause one category of connection issues. It is known, understood, and fixable -- but the fix involves releasing a new version of the Driver Station application and modifying to the default Dashboard project. Working around that problem is simple enough that it's probably not going to get fixed in this season's code.
The true "quirks" are most likely due to background tasks running on a team's Driver Station computer. FIRST can't do anything about that. It's up to the teams to clean up their computers' configuration, turning off antivirus scanners and firewalls and software updaters and streaming music clients and Skype and everything that isn't part of the FRC software.
VioletElizabeth
06-04-2015, 11:19
SVR was actually ahead of schedule a lot of the time. (I know because it really messed with our practice field slots.) The theory I heard for this was because the announcers were announcing teams during field reset. However, there were sometimes the aforementioned connectivity issues. Whenever I noticed a delay, that was the problem. Of course, I saw maybe 25 of the 95 matches because I was in the pits, so sample size is not perfect.
Jeremy.F
06-04-2015, 11:46
In Indiana at the state championship tournament, the field was running nearly a match ahead of schedule (8 min ahead). This can be counted towards in a district it was everyone's 3rd event and it was the same field crew at most events. Teams were complaining to the point that we had to do delay songs/dances when there was no problem at all.
So back to my original concern...do you think I will be safe planning for 8 minute turnaround...as long as we can keep field reset, and robot-transport configuration-transition within reasonable times as well? I feel comfortable predicting physical setups...just have no idea what to expect with the networking resets.
I am assuming that by October, more of these quirky connection issues will be figured out and solved. I want to figure out how "big" of an event we can plan without running too long or sacrificing number of games played per team.
I was the FTAA at the Colorado Regional a couple of weeks ago. We scheduled 7 minute turns for our matches. We ran behind Friday morning as we spent time getting all of the times to configure their driver stations correctly. However, once we got all of that figured out with the teams we were able to keep to our 7 minute schedule for the rest of the weekend.
As far as team experience (first event vs second/third event), I don't have the exact numbers handy at the moment but I believe that roughly 1/3-1/2 of the teams had already attended an event before coming to ours.
8 minute turns is definitely doable and should allow for a slightly more relaxed atmosphere and give you some more wiggle room.
Justin Montois
06-04-2015, 13:06
Team #: jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj
plnyyanks
06-04-2015, 13:11
So back to my original concern...do you think I will be safe planning for 8 minute turnaround...as long as we can keep field reset, and robot-transport configuration-transition within reasonable times as well? I feel comfortable predicting physical setups...just have no idea what to expect with the networking resets.
I am assuming that by October, more of these quirky connection issues will be figured out and solved. I want to figure out how "big" of an event we can plan without running too long or sacrificing number of games played per team.
I was the FTAA at the Colorado Regional a couple of weeks ago. We scheduled 7 minute turns for our matches. We ran behind Friday morning as we spent time getting all of the times to configure their driver stations correctly. However, once we got all of that figured out with the teams we were able to keep to our 7 minute schedule for the rest of the weekend.
This is consistent with what I saw at two events. Both scheduled 7 minute cycle times, ran ~20 minute behind Friday morning, ~10 minutes behind Friday afternoon, and basically on time Saturday morning. Once the field hit its stride, we averaged between 7 and 7:30 cycle times. So, you should be fine with 8.
akoscielski3
06-04-2015, 13:15
I'm at Windsor. How do you tell what field is here?
I volunteered at Windsor as a FTAA
Field 12 was the Windsor Field. The field was also previously in Montreal and North bay. It will not be at championships though.
There was a lot of connection issues and replays in Windsor, we believe this was due to the venue and the amount of WiFi/Hot Spots in the area.
When the field was working properly on Saturday morning we were running 6 minute cycles (for the most part).
Hopefully next year we do not have similar issues!
Also volunteering as a FTAA was a lot of fun, and a great learning experience. I would recommend it to anyone (however it is very tiring and a lot of work)
The "red case" field was working pretty well for us at AZ West.
It was actually kind of fun to have to tell teams a few times throughout each day, "Do NOT power on your robot just yet. We have to let the schedule catch up to us a little bit. We are getting too far ahead and teams are not having their rightful chance to repair/change things before their next match is queued."
I'm not sure that I can get used to being able to sit down for a leisurely lunch in the middle of the event days though. That is just plain weird. That is supposed to be a time of chaos...or it has been in my past experience of several years of FTAA duty.
Thanks to all of the Arizona West teams that made the event a sheer pleasure to be your FTAA!!
Also volunteering as a FTAA was a lot of fun, and a great learning experience. I would recommend it to anyone (however it is very tiring and a lot of work)
Want to have some fun with being the FTAA? Wear a device like a Bodymedia (my favorite for the parameters it keeps track of) or a Fitbit to see how much you move. Seeing the numbers at the end of each day just makes you go, "Ouch!" Especially when you realize that it was mostly done in an area the size of a tennis court.
Want to have some fun with being the FTAA? Wear a device like a Bodymedia (my favorite for the parameters it keeps track of) or a Fitbit to see how much you move. Seeing the numbers at the end of each day just makes you go, "Ouch!" Especially when you realize that it was mostly done in an area the size of a tennis court.
...and mostly on concrete (depending on venue)
plnyyanks
06-04-2015, 20:41
...and mostly on concrete (depending on venue)
FTAAing at convention center regionals crazy hard on your feet. Just thinking about NYC is making mine hurt again.
That being said, FTAA is definitely my favorite volunteer role, but it's not easy at all (or stress-free)
Qbot2640
06-04-2015, 21:03
The true "quirks" are most likely due to background tasks running on a team's Driver Station computer. FIRST can't do anything about that. It's up to the teams to clean up their computers' configuration, turning off antivirus scanners and firewalls and software updaters and streaming music clients and Skype and everything that isn't part of the FRC software.
This is the kind of information I was looking for. We can plan ahead, to have someone who is FMS-savvy visit the teams and "advise" them on their driver stations. An hour spent on Saturday morning to help the event run smoothly all day would be a worthwhile investment.
This typically happens when teams are plugged in before the field has completed prestart (and is usually worse on practice day, when we're making substitutions.
I think the faster turnaround time from the refs as they hit their groove (so the field can complete prestart before the next round of teams plugs in) is the biggest factor in these delays not being seen later in the competition.
Again...great information. Process each end of game efficiently, and have the next teams do everything else before they plug in.
Thanks everyone!
plnyyanks
06-04-2015, 21:38
This is the kind of information I was looking for. We can plan ahead, to have someone who is FMS-savvy visit the teams and "advise" them on their driver stations. An hour spent on Saturday morning to help the event run smoothly all day would be a worthwhile investment.
I've found the worst offenders of DS background software include: Autodesk 360, Dropbox/Google Drive, Spotify, and the like. You should also make sure that any firewall/antivirus is totally disabled (or exceptions for the Driver Station program and NI mDNS Responder)
Joe Ross
06-04-2015, 22:06
This is the kind of information I was looking for. We can plan ahead, to have someone who is FMS-savvy visit the teams and "advise" them on their driver stations. An hour spent on Saturday morning to help the event run smoothly all day would be a worthwhile investment.
If there isn't a 2015 FMS lite, you may need to visit every team's pit based on what Networking Configuration (http://wpilib.screenstepslive.com/s/4485/m/24193/l/308349-running-a-2015-week-zero) you choose.
It's up to the teams to clean up their computers' configuration, turning off antivirus scanners and firewalls and software updaters and streaming music clients and Skype and everything that isn't part of the FRC software.
I would argue that it's up to FIRST to let teams know that they need to do these things. Little tricks, like disabling background processes, virus scans, bluetooth drivers, and removing power saving features on USB aren't widely used because teams don't have this information readily available.
akoscielski3
07-04-2015, 08:44
Want to have some fun with being the FTAA? Wear a device like a Bodymedia (my favorite for the parameters it keeps track of) or a Fitbit to see how much you move. Seeing the numbers at the end of each day just makes you go, "Ouch!" Especially when you realize that it was mostly done in an area the size of a tennis court.
I have Google fit on my phone, here are my steps:
Thursday: 28 235
Friday: 27 628 (actually more since phone was dead for a long time)
Saturday: 28 131
And I agree with FTAA being a really fun role. I did score keeping last year and I really enjoyed that to (although there would have been a lot less steps per day).
I have Google fit on my phone, here are my steps:
Thursday: 28 235
Friday: 27 628 (actually more since phone was dead for a long time)
Saturday: 28 131
And I agree with FTAA being a really fun role. I did score keeping last year and I really enjoyed that to (although there would have been a lot less steps per day).
Yeah, those are in the ballpark for mine as well for all of my FTAA gigs (7? at this point). My stride is 2500/mile which would be ~34 miles for your numbers (let's assume that even if it isn't step being recorded that it is probably a down & up cycle trying to find the radio/roboRio and equivalent to a step).
I still say being the FTAA is the most exhausting fun I've ever had...and I love doing it. Bring more on! Oh yeah, the Arizona State AIA Championship is my next FTAA assignment...ROBOTICS AS AN OFFICIAL STATE RECOGNIZED SPORT!!
vBulletin® v3.6.4, Copyright ©2000-2017, Jelsoft Enterprises Ltd.