[FRC Blog] Einstein Report Released

[PayneTrain]

The organization did what they were expected to by members of the FRC community and more, even with the news arising that intentional interference took place. my eyes really widened when I read that. I'll finish it after I let that thought sink in.

It would have been a poor option to take away the award from the kids and their organizations that support them, but it's great that they extended the berths to all teams on Einstein and paid for the teams to register in 2013.

[Akash Rastogi]

I think it is important to remind readers to not be too shocked and awed by the fact that interference happened; I think it should be viewed as another thing that is potentially a problem with the control system.

My point being; don't let this news be the face of the issues with the control system. The fact that these elite teams were able to have such problems with the system leads me to say that perhaps this is not the ideal system for FRC.

This report is great and revealed many problems teams may have dealt with all season.

+0.02

[bearbot]

WOW thanks you to FIRST for releasing this out to everybody I appauled that it hopefully wont happen again.We had major issues with the FMS @ Cheaspeake regional and saw they were used on Einsten field to.Whish FIRST would have done something to the fields that the electronics came from to instead of just from the Einsten matches

[Brandon_L]

Originally Posted by SenorZ

Hmmm. Intentional interference... sounds like FRC is really starting to look like a "varsity" sport.

Aside from Einstein having issues there were tons of radio and cRIO issues at regionals. It really is time for FIRST to look at a new system.

After reading the report, I think I could say that I now support FIRST's claims of "Its not us, its you" in regards to comms loss at the regionals this year. If 5 of the 12 Einstein teams could have electrical/programming issues, I can see it being a larger problem then I thought before. Even though most of the errors were silly mistakes that wouldn't effect comss, imagine all the other teams that could make silly mistakes that would. The control system is fine, if put together correctly. More documentation is all you could really do about that, but teams have to read it. Now the radio, on the other hand..


I'm impressed with the detailed report and the handling of the situation by FIRST. Can't wait to get my hands on the FMS white paper!

As to the "interferer", I see his actions "accidentally intentional". From my understanding of the report, he simply tried to connect to the robot and provided an incorrect network password. I've seen robot networks on my mobile device many times, and tried connecting to it in the shop out of curiosity. Nothing really happened in that environment, but I'd give him the benefit of the doubt and call it a case of curiosity killed the cat. After all he did come forward.

[davidthefat]

ಠ_ಠ So, asides from the intentional foul play, it seems as if their was a lot of errors on the teams themselves. Seems like 987's problem was that there was a deadlock with their software. So, that's a shame.

[Racer26]

Wow. Intentional interference. Shame they decided to let the results stand after that. I'm sure 1114/2056/4334 are left with a rather sour taste in their mouths.

I read the whole report, and while a number of possible issues were identified for the various teams, only 118's dropouts were really confirmed to be caused by team equipment/configuration.

I will say that for the FCA attacks on 1114/2056/4334 to be intentional, someone would have had to have figured out that the FCA vulnerability existed, and based on the way the report talks of it, it seems that this only existed from Wk4->Championship, and only on robots with rev A hardware. Curious, indeed.

I still stand by my original assessment from April that the proper way for FIRST to have handled Einstein is to award all 12 teams 2012 FRC Champions, in lieu of being able to play a fair set of matches.

[Steven Donow]

Originally Posted by 1075guy

Wow. Intentional interference. Shame they decided to let the results stand after that.

Barry Bonds used steroids, however in the record books he is still listed as hitting the most amount of homeruns.

Just thought I'd throw that in for comparison. Also, it isn't known(well, to the general public, i'd assume those involved do know) whether or not the interferer was directly targeting a specific team; just that he was only SEEN targeting 2056 directly.

IMO intentional interference being reasoning to change the results is an invalid point. As stated, FIRST is now more like many varsity sports

Also of interest was that it seems like 118's issue was also code and that no team on Einstein was using Labview. For reference, do 1717 or 469(had comm issues in their semis) use labview?

[R.C.]

Originally Posted by stevend1994

For reference, do 1717 or 469(had comm issues in their semis) use labview?

1717 Did not use labview for sure.

Originally Posted by 1075guy

Worth noting: I expect the FRC Community at large to be wholly unsatisfied with HQ's decision not to replay the matches (and/or declare all 12 teams Champions in lieu), given the circumstance of intentional foul play.

I don't think you can blame FIRST for not replaying the matches... Replaying Einstein 2012 would be asking for trouble. Teams or the community shouldn't be unsatisfied. The alliance of 25/180/16 were darn solid. 25/180 hit their shots and could triple. 16 was by far too freakin amazing at stealing/feeding. And the alliance had a really solid auton/endgame.

Teams and the community should be stoked that FIRST took a giant step forward and are working towards figuring out issues with the control system. They even published a very nice paper. Kudos to FIRST.

-RC

[Radical Pi]

Originally Posted by 1075guy

I will say that for the FCA attacks on 1114/2056/4334 to be intentional, someone would have had to have figured out that the FCA vulnerability existed, and based on the way the report talks of it, it seems that this only existed from Wk4->Championship, and only on robots with rev A hardware. Curious, indeed.

I have a feeling that while there was intentional FCA going on, there's a large potential for unintentional FCA as well, which they didn't mention whether or not had been tested. Many teams configure their practice network at the shop with the same SSIDs that the field uses, and if they used a different security key, any phones/tablets/laptops that happened to be active during the games and had been connected to the network at home could have been attempting connections in the background. I actually caught our team's programming laptop trying this once during week 2.

Also, I was surprised to notice this year that SSID broadcast was enabled on the field APs, making the team networks show up on every laptop in range. I don't remember it being that way last year.

On a somewhat related note, what would people think about having a CSA check over code as part of inspection to counter the kind of user code issues that were seen in the report. It wouldn't have to be a pass/fail kind of thing, but having a more well-trained pair of eyes to provide suggestions to the team and something for field staff to go on when a robot misbehaves would probably go a long way

Anyways, excellent report by FIRST, and I think bandwidth caps are a great idea. I hope they don't take the partial involvement of the BeagleBone as a reason to ban non-KOP electronics though. They really open up new possibilities for teams.

EDIT:

Originally Posted by stevend1994

Also of interest was that it seems like 118's issue was also code and that no team on Einstein was using Labview. For reference, do 1717 or 469(had comm issues in their semis) use labview?

I don't know about those two, but after those semis 68 had unexplained comms issues in the finals, and they do use labview

[Jon Jack]

I think FIRST has handled the situation well so far. FIRST could have easily turned a blind eye and tried to sweep the problem under the rug. What would replaying the 2012 championship matches really solve? Nothing. In my opinion, giving the 12 teams automatic invites to the 2013 Championship as well as waiving their first play registration is enough consolation.

Originally Posted by Radical Pi

there's a large potential for unintentional FCA as well, which they didn't mention whether or not had been tested. Many teams configure their practice network at the shop with the same SSIDs that the field uses, and if they used a different security key, any phones/tablets/laptops that happened to be active during the games and had been connected to the network at home could have been attempting connections in the background. I actually caught our team's programming laptop trying this once during week 2

The issue isn't just trying to connect to the network, the issue is trying to connect to the network - while entering an incorrect WPA key:

During the Post-Championship field testing an attempt was made to connect to one of the team SSIDs set up on the field network, but the WPA key was entered incorrectly. This was observed to sever the communication
between the driver station and robot associated with that SSID. After further testing, a link between failed authentication attempts and a disruption of the communication between a driver station and robot was confirmed, though not every failed authentication attempt resulted in a communication disruption.

Lets hope that FIRST continues to learn from Einstein 2012 and makes changes to the control system to prevent this from happening at future events.

[Ekcrbe]

Originally Posted by Radical Pi

I don't know about those two, but after those semis 68 had unexplained comms issues in the finals, and they do use labview

I'm not entirely sure about that incident, but I had a sneaking suspicion even before Worlds that we were somewhat vulnerable to dropouts when being hit near the battery. It could have been that, considering it coincided with a collision right there with the corner of another robot.

Plus, it was only once, and the NI experts in the pits actually reviewed our code and found it to be fully functional and efficient enough (~50% CPU usage average).

[JB987]

Originally Posted by davidthefat

ಠ_ಠ So, asides from the intentional foul play, it seems as if their was a lot of errors on the teams themselves. Seems like 987's problem was that there was a deadlock with their software. So, that's a shame.

David,
If you read the full report carefully and pay close attention to the following:

987 – While testing of this robot revealed programming issues that could cause higher than normal trip
times, or the cRIO controller user code to lock up, none of these issues were found to cause a command response failure. Which was the cause of our "dead" time on the field). The duration of these losses were too short to be attributed to a cRIO or robot radio
reboot, but fit well with the symptoms of failed client authentication.

You will see that we too were likely victims of the intentional"failed client authorization" debacle...

[davidthefat]

Originally Posted by JB987

David,
If you read the full report carefully and pay close attention to the following:

987 – While testing of this robot revealed programming issues that could cause higher than normal trip
times, or the cRIO controller user code to lock up, none of these issues were found to cause a command response failure. Which was the cause of our "dead" time on the field). The duration of these losses were too short to be attributed to a cRIO or robot radio
reboot, but fit well with the symptoms of failed client authentication.

You will see that we too were likely victims of the intentional"failed client authorization" debacle...

Yes, I admit, I have not read carefully enough. Still, that's a shame IMHO. A deadlock anywhere is not good.

[Racer26]

Worth noting: I expect the FRC Community at large to be wholly unsatisfied with HQ's decision not to replay the matches (and/or declare all 12 teams Champions in lieu), given the circumstance of intentional foul play.

[Hjelstrom]

Originally Posted by davidthefat

Yes, I admit, I have not read carefully enough. Still, that's a shame IMHO. A deadlock anywhere is not good.

David, the deadlock is within the WPILibrary and we have been helping them resolve it. In fact, just today we were at the shop with our programmer Brandon as he finished up test programs for FIRST to use to reproduce two bugs we encountered in the smartdashboard support code this season.

These bugs did not affect our robot on Einstein or in any match all year because we avoided them. Working towards getting them fixed for next year is just a side benefit of the New Hampshire meeting.