Einstein 2012

[RyanN]

Originally Posted by stevend1994

Excellent analogy with the light waves, but my question in response to this theory (and I'm nowhere near an expert, or knowledgeable at all in this field, so it's not rhetorical, it's an actual question) is if it happened so strongly at this level on Einstein this year, why has it not happened in the past? Wasn't last year's control/field system as a whole(minus the Kinect) the EXACT same components as this season?

True...

There was a lot more data being transmitted this year with the camera interface added this year. More data to be decoded properly. That doesn't help us in our situation at Bayou though. We went back to stock code, no camera, no CAN, and still couldn't connect.

I also know that the smart phone market has been exploding for the past few years. I actually can't name any one of my friends, and almost every smart phone has WiFi onboard. I know, at least with my iPhone, if I have the "Ask to Join Networks" switch on in the Wi-Fi menu, it will scan and ask me to join some networks. This creates network congestion, but no one notices it normally because there's not network jam, but get over 1000 people, with say 50% of them carrying around a WiFi enabled smart phone, and I would say you might run into some trouble.

I honestly have no idea... it's all speculation until FIRST comes out with an official statement. It might be a configuration problem on their part with FMS. Who knows for sure?

[RyanN]

Originally Posted by sanddrag

I am no expert in radio waves, but I can say we've experienced the same thing. We ran our robot in the school's talent show. We did a trial run with nobody there with great connectivity from probably 70 feet away. In the actual event, we had about 900 students all presumably with cellphones (on), and we were struggling with latency and packet loss even over a 30-foot distance to the robot. There has to be some correlation.

One of the things I want to know is, of all the teams that didn't work at some point on Einstein, what code language were they running?

LabVIEW

[NickTosta]

Originally Posted by sanddrag

One of the things I want to know is, of all the teams that didn't work at some point on Einstein, what code language were they running?

I'm fairly certain that 118 uses C++. I think the problem is independent of the programming language being used.

One random theory that hasn't been thrown out there yet - I wonder if all of the teams that had issues are using the CAN bus? Or maybe they are all teams that are using all 4 ports on their router? Or maybe they are all teams that are using the serial port on the cRIO?

[RyanN]

Originally Posted by NickTosta

I'm fairly certain that 118 uses C++. I think the problem is independent of the programming language being used.

One random theory that hasn't been thrown out there yet - I wonder if all of the teams that had issues are using the CAN bus? Or maybe they are all teams that are using all 4 ports on their router? Or maybe they are all teams that are using the serial port on the cRIO?

We tried with and without CAN. No difference. I'm fairly confident that this isn't a cRIO or a software issue. If it was, these teams would have also seen this issue on their pre-Einstein fields.

[techhelpbb]

I've asked this elsewhere but the D-Link is a dual band AP. If it's listening on 2.4GHz, even if the field is in the other band, 2.4GHz connection activity could interact with the radios.

Normally in a WiFi network a dual band radio would offer bridging of all the traffic on the switch with all the traffic authorized on both bands. The only way to stop that behavior is disable one band.

It could be that in a room with a very large number of 802.11G networks just the scanning past that other band could cause issues if the 2.4GHz radio portion is not disabled.

Also to the conversation directly above at MAR Mount Olive (I was in spare parts) one team having major communications problems was using Labview and CAN. We discussed the possibility of the issue being the CAN, but it didn't seem likely. Someone did use my oscilloscope to check their power, but in the pits not on the field. We also considered that Labview might not trap exceptions quite the way other languages do so. One should keep in mind that with so many language options it's entirely possible that Labview has one communications effecting issue and C++ has others. Some examples of communications issues effecting teams running C++ were noted elsewhere in the forums.

There are so many possible combinations of problems. Hard quantifiable evidence will be needed. Now the problem for FIRST will be that Championships are over and the effected robots might not be in the states they were at the time of the issues when and if they appear at other events. In point of fact, neither will the fields after they are taken down, moved and reassembled elsewhere.

[ticoloco12]

Hello all,

Now that the dust has settled, if only a tiny bit, I'll share what I saw from Einstein. You may know me as Will, I work with team 207's strategy and drive team. I am a FIRST Alumni and have been mentoring for 4 years now. let me make it clear that Team 207 is home and nothing but thankful to team 233 and 987 for an amazing time at world champs and our first trip to Einstien. Through the chaos that was the Finals, they showed that they are pure class and of the highest examples FIRST has. (341, 254,and 78 as well)

Yes, many, an unforgivable number in fact, of robots were either dead or experienced significant periods of losing coms. Some, clearly game changing. This sucked the emotion and prestige of the finals, clear out of every team on the field. Not one of the drive teams, or pit crews in our alliance cheered as we made it into the finals after semi's. It was a feeling of disbelief, that FIRST was allowing such a thing. It would have taken 30 minutes, to replay all of Einstein back to back in my mind. but that did not happen.

As a programming mentor for Pink said earlier, this is not how ANY FIRST team wanted to win or lose. I remember being on the drive team myself and dreaming of one day coming back to mentor my team there, to feel that rush, to cheer to the absolute maximum the human lungs can. To have been dead silent, and not even be able to rally our team between F1 and F2 because everyone's sentiment was, whoever works longer will win, is the largest disappointment I have ever had in my FIRST life. Even if making it there after an EPIC curie division Finals. From what I see in finals videos and scores, 341's alliance in the division finals was as competitive if not more so than any alliance on Einstein.

However, the most important feeling and experience to get away from this, is the spirit of FIRST teams. They are completely separate from fields and external chaos such as sever weather, curses, coms issues and all else. Every team on Einstein was united in their dislike of what happened, and treated each other graciously and professionally. When 987 needed time to inspect their robot and wanted to call a timeout (and I still don't know why we were denied one) 180 came to offer any assistance and tried to call theirs as well. both finalist alliances congratulated each other, on the field, and in the pits. The students were proud and even 118, and the Canadian teams did not hesitate in spreading hugs and handshakes to us all. The students themselves began making plans to replay all matches at IRI with robots in the same condition as they left the field Saturday. Dean Kamen was on the field, as was Dave and the look on their face said it all. They were disgusted and Dean was almost in tears as a Girl from a team I cannot remember pleaded the cause of her team. Dave was gone it seemed by the time the finals were over, and Woodie's face from the stage looked like he was viewing somebody drown and could not help. I insist, the truest qualities of FIRST were never seen in such bright colors and they were here. Technology failed us and perhaps the field crew have a huge catastrophe to correct. These finals will be remembered as the day the field chose the winner. BUT IN NO WAY, SHOULD THESE EVENTS TARNISH THE IMAGE OF FIRST, ITS MISSION, ITS STUDENTS, TEAMS, OR FOUNDERS.

Some of the greatest Robotics matches I have ever seen took place on Curie field in the finals. And I speak for every member of team 207 when I say we are beyond proud of all we did, and all our alliance partners did. We would put money on our alliance anyday, and if we lost a fair match, we'd cheer so loud for the alliance that beat us that our lungs would explode. Regardless of what trophies or banners the fields gave us, the embraces, laughs, comments, and connections with amazing FIRST-ers are sooooooo much more valuable. I truly hope the Einstein replay at IRI happens, not to play these matches, as much as to share them with such amazing teams and individuals, like we should have been able to.

I hope a lot changes next year, but only as far as the field goes with technology, and in organization and error management. the Human aspect of FIRST and it's competitors is something we can be prouder today of than ever.

-Will

[Mulcahy]

I would like to thank Dean and Woodie for the amazing experience the last few days have been. If we came in 1st or 400th, the fact that we were able to participate in such an event is truely remarkable. Woodie even delayed getting on a plane to sign a shirt for my son. Thank you for everything!

-Steve and Brendan from team 987

[Thad House]

I think that if FIRST can figure out a way to get the robots back to HQ with the field as well, and maybe get some of the students over to HQ they might be able to work this out better then just looking at the logs alone. I am sure that none of the robots have left St. Louis yet, and if FIRST gets in contact with the teams soon enough, they might be able to reroute the crates to HQ. I am sure most of the teams would be willing to do this, seeing as we have multiple HOF teams that show that they are all about helping FIRST.

[qzrrbz]

After 40 years of dealing with software problems, it is my gut feel here that we are chasing one of the worst nightmares of any troubleshooter -- a problem that is *data* dependent, *environment* dependent, and probably intermittent at best.

A terrible thing to try to reproduce, understand, and solve! Good luck to all involved!

[kylelanman]

I was there and after the second match on Einstein myself and the others I was with were not watching the robots during the match. We focused our attention on the driver station lights. Particularly at approximately a minute left it was inevitable that one or two would start flashing. My team dealt with connectivity issues in WI in 2010. We made it through F1 undefeated and then F2 and F3 comms dropped right after switching to teleop. Field personal told the students "Your comms were in a tizzy" As others have said I'm really hoping that because this finally happened on Einstein that FIRST will deal with the issues.

Originally Posted by cgmv123

The FMS access point is a CISCO product. The Control System advisor that helped us with our one hangup said it was ~$10,000. I don't know the model number. I'm trying to find out, but I have nothing so far. It's about the size of the D-Link, but with 6 antennas sticking out the top.

Edit: the one place I thought the details would be doesn't have it. All the Field setup manual says is "Field Access Point" and "Cisco radio". It does have an image though. Still looking.

I'm going purely off the looks but the AP looks like a Cisco AIR-AP1252 with a wireless G and wireless N modules installed. We have these same access points were I work and they run about $900. My experience is they are a rock solid device.

In the little information I have found on the FMS (FTAA Training section of the FIRST site) there is also a Cisco router involved not sure of the model number there.

Multiple people have indicated that the FTA has told them to wait until the robot is on the field and there team number is displayed before powering on the robot. My guess is that each team is on there own SSID and VLAN. If the robot connects prior to the correct VLAN being configured on the fly then all the devices are not able to take part in the initial discovery phase and the ARP cache is then lacking key entries. I'm guessing/assuming the router is reconfigured in between each match automatically by the FMS. My theory is that the FMS configures the AP SSID first and then the routes on the router. If the robot connects to it's SSID but the route is not yet configured for it to speak to the driver station and possibly the FMS server then problems arise.

With all that being said I could see it taking a while for the robot to connect or for it to not connect at all. But I can not think of how the above scenario would cause it to die a minute or so into the match.

Any FTA's or FTAA's out there that care to weigh in on this?

[Azores]

"Dear FRC Teams:

Thank you for your incredible enthusiasm and Gracious Professionalism throughout the year and at the Championship.

We apologize for the technical problems that affected the final matches at our Championship. We will examine all of the facts, report our findings and ultimately solve any and all identified issues.

Sincerely,
Jon Dudas
President, FIRST "

Just received this, posting it here in case anyone doesn't receive it.

[AlecMataloni]

I couldn't even watch the matches. My eyes were glued to the alliance station lights.

My thoughts go out to all the teams affected by this mess.

[hiyou102]

http://redd.it/syry2
This.

[techhelpbb]

Originally Posted by hiyou102

http://redd.it/syry2
This.

I would like to ask as a Mentor that while I, possibly more than a lot of people, deeply understand the concerns about what happened here.

Please, please, please think about the consequences to FIRST in the future with regards to things you write today.

FIRST is a wonderful organization that has done a lot of very good things. However, there are people that have issues with FIRST that would love to extend the single failure to troubleshoot into evidence of a pattern.

Just remember that when you remove your posts from the vacuum of ChiefDelphi you run the risk that they might stick around longer, attract more attention, and even that you might not be able to go back and close the matter when the issues do get resolved.

I am not saying that what I read at that link is disrespectful, just please realize that the bigger FIRST gets the bigger the politics around it get.

[frdrake]

Originally Posted by RyanN

LabVIEW

We ran C++ this year.

I also want to thank 987 and 207 for their tremendous effort in their part for our alliance this year. I know we couldn't triple, but we each had the right pieces to bring to the table to maximize our strategy. Thank you both for the maturity of your drive teams and enthusiasm to play.