[FRC Blog] Einstein Report Released

[jason701802]

Originally Posted by connor.worley

Issues in years prior may have had nothing to do with the deauth attack- only one radio and one AP that they tested were vulnerable, the ones used this year.

And that is precisely why this report fails at its root. I care very little about the problems that affected 12 teams when there are other problems with the control system that affect 100s of other teams.

[Jibsy]

Originally Posted by Nate Laverdure

If I were the boss, I wouldn't have released this report on Friday the 13th.

This made me laugh in the midst of being upset about the news. Thanks for that haha.

[jblay]

Originally Posted by Basel A

While I don't think there's a strong case for replay or any change to the officially announced results of the 2012 FRC Championship, I do think it's worth noting that prior to their being knocked out, the Archimedes alliance was the only one suffering problems likely caused by FCA. If the interferer had an agenda, it seems that 1114, 2056, and 4334 was the primary target.

I think this thought process is very dangerous, before you know it you are pointing the finger at certain teams that were on Einstein. It could have been so many iterations of people and pointing the finger at someone specifically the wrong someone is not what FIRST is about.

[KrazyCarl92]

Originally Posted by Alexa Stott

If you read the document carefully, it states:


Later on, it states that the likely cause of 1114's issues was a sensitive main breaker.

Originally Posted by FIRST Einstein Investigation Report Page 16

Based on these results, it is unlikely that the sensitive main breaker would have caused command response failures during the actual Einstein matches.

It says the exact opposite of that.

[Alexa Stott]

Originally Posted by KrazyCarl92

It says the exact opposite of that.

Yeah, my bad. I fixed my post.

[Karthik]

FIRST Hall of Fame Team 1114, Simbotics, would like to thank FIRST for thoroughly investigating, addressing and documenting the robot failures that took place on the Einstein Field at the 2012 FIRST World Championship. We would especially like to thank Frank Merrick and his staff for constantly communicating with us through the process.

We are shocked, dismayed and troubled that an individual on a FIRST team would actually perform an intentional, malicious, wireless attack on our alliance. We are concerned that neither the individual nor the team he is associated have yet to come forward and publicly apologize for this horrendous incident. We hope that they come forward publicly soon, so we can all put this terrible event behind us. It would be a shame if they hid under the cloak of anonymity. Even if the team was completely unaware of the individual's actions, we would still hope that they would come forward, so that some of the motives would become more clear.

Words can't express how much this news hurts. To know that someone felt the need to intentionally target us for this type of attack stings beyond all belief. This is not the FIRST we grew up in, this is not the FIRST we love.

[Billfred]

When I saw the email blast come through on my phone, I seriously sat in my car for 20 minutes or so reading the PDF. My thoughts:

1) I applaud the team for being this thorough in their methods and sharing all the steps they took (and for ruling out things we would normally hold up as the cause).
2) I believe FIRST has done right by the teams involved. Nothing is gained by replaying the matches.
3) I thank the teams for checking their own systems and code to discover issues of their own.
4) Shame on the individual who attempted this stunt. It hurts that FIRST has to have its answer to baseball's permanently ineligible list. We should all expect a higher standard.
5) I hope some element of this process is framed and put somewhere fairly prominent in FIRST Place. I think this entire story contains elements of the FIRST experience both at its worst (see point 4) and at its best (see point 3). Much can be learned from both.

[CalTran]

That was one well written report, and certainly insightful to the whole FMS process. It gave our team a thing or two to look out for in the up coming seasons, as well as possible things to look immediately at for debugging.

As far as the repeated notions of a "tainted" win and questions of replay or total recall, I believe that is out of the question. 16, 25, and 180 won the 2012 FIRST Robotics Competition World Championship, and should not be forced to defend their title.

[dodar]

Originally Posted by Karthik

FIRST Hall of Fame Team 1114, Simbotics, would like to thank FIRST for thoroughly investigating, addressing and documenting the robot failures that took place on the Einstein at 2012 FIRST World Championship. We would especially like to thank Frank Merrick and his staff for constantly communicating with us through the process.

We are shocked, dismayed and troubled that an individual on a FIRST team would actually perform an intentional, malicious, wireless attack on our alliance. We are concerned that neither the individual nor the team he is associated have yet to come forward and publicly apologize for this horrendous incident. We hope that they come forward publicly soon, so we can all put this terrible event behind us. It would be a shame if they hid under the cloak of anonymity. Even if the team was completely unaware of the individual's actions, we would still hope that they would come forward, so that some of the motives would become more clear.

Words can't express how much this news hurts. To know that someone felt the need to intentionally target us for this type of attack stings beyond all belief. This is not the FIRST we grew up in, this is not the FIRST we love.

Even though I think that the teams that had this happen to them deserve to know who did it and why, I foresee what you are asking turing into a similiar mindsight as the Jerry Sandusky case. That is, that the perpetrator did it full knowing what he was doing and what the consequences could be and forgoing the effect it would have on others, but it in turn not only effected him but also Penn State; the point im getting at is, putting forth the person and team publicly would do far greater harm to the team and their image amongst their peers and the FIRST community. I think a much better solution to what Karthik wants, and is justified in wanting, is to have all the parties involved meet so that everyone can be satisfied as to reasons why, punishment, etc... But I do beleive that this shouldnt be known to the masses.

[techhelpbb]

The larger issue than who did this is how was the system allowed to get to the point that it was possible at all. Let's worry about all the other D.O.S. (denial of service) attacks we have yet to find.

Clearly changes need to be made. It took extraordinary effort on the part of too many people to resolve even these issues to this point.

It still goes back to the assumption that the system is above flaw and that assumption being incorrect. In this case the system has a security issue and an active exploiter. Take if from me: you can look for and fix security issues before they get exploited as best you can or you can wait until they cost you reputation, resources and opportunity. Had they even profiled the issue before hand they could have dramatically reduced the chaos after the fact (if you don't fix it at least acknowledge it exists at the remediation level).

FIRST needs to consider a secondary channel in their control system if they can't more fully profile something like the WiFi system they have now. It's the logical alternative to pumping all the data into a single publicly exposed communication system. FIRST did have the AirTight equipment on site but clearly that alone wasn't sufficient to keep a lid on this issue. Additionally the FIRST report oddly doesn't discuss that the AirTight hardware did not produce for them sufficient warning of this issue or whether they consider that something they need to pursue (it was the assumption that this additional monitoring was sufficient to keep intentional interference from happening at all).

I give FIRST credit for the heavy work they put into this. I'm just not convinced that this won't happen again if someone intends to deny service to the field. Nothing I've seen in the recommendations will stop it. I can think of 3 ways right now that if I wanted to render all the field robots dysfunctional I could and it would be nearly impossible for them to discover it. Even if the sudden failure was recognized as intentional disruption the cost in resources to weather such interference is unacceptably high. A better solution needs to be found for this.

[Alexa Stott]

Originally Posted by dodar

Even though I think that the teams that had this happen to them deserve to know who did it and why, I foresee what you are asking turing into a similiar mindsight as the Jerry Sandusky case. That is, that the perpetrator did it full knowing what he was doing and what the consequences could be and forgoing the effect it would have on others, but it in turn not only effected him but also Penn State; the point im getting at is, putting forth the person and team publicly would do far greater harm to the team and their image amongst their peers and the FIRST community. I think a much better solution to what Karthik wants, and is justified in wanting, is to have all the parties involved meet so that everyone can be satisfied as to reasons why, punishment, etc... But I do beleive that this shouldnt be known to the masses.

The PSU case is different-many university officials actively participated in a cover up. I don't think any comparisons should be drawn between the two in any case, given the difference in severity.

[kjohnson]

Originally Posted by techhelpbb

Additionally the FIRST report oddly doesn't discuss that the AirTight hardware did not produce for them sufficient warning of this issue or whether they consider that something they need to pursue (it was the assumption that this additional monitoring was sufficient to keep intentional interference from happening at all).

I know the report is long, but I urge everyone who wants to make a statement about the report read the entire report first. There have already been multiple posts in this thread containing false assumptions.

Originally Posted by Denial-Of-Service Attacks p.10-11

...
The AirTight system successfully detected and flagged this test as a “Denial of Service Attack” event.
...
The next tests involved targeting the attack at the specific client (the robot radio) and throttling the rate at which the tool sent the de-authentication packets. These attacks were also successful at disrupting the communication, but were able to elude detection by the AirTight sensor. Further investigation into the thresholds for classifying an event of this type in the AirTight system revealed a minimum of 8 directed or 4 broadcast de-authentication packets per second for a minimum of 90 seconds must be detected for the AirTight system to flag and classify a denial of service event. FIRST has determined that these thresholds provide insufficient protection for the FRC wireless use case.

Originally Posted by Next Steps, p.23

A number of solutions to the network susceptibility to a “De-Auth Flood” attack will be investigated including working with AirTight to modify thresholds for detection, implementing custom capture and detection of these attacks, and changing radios to a custom solution which is more resistant to these attacks.

[bardd]

Originally Posted by 1075guy

Worth noting: I expect the FRC Community at large to be wholly unsatisfied with HQ's decision not to replay the matches (and/or declare all 12 teams Champions in lieu), given the circumstance of intentional foul play.

If one of the winning teams had to do with this interference, or even any of the teams who played in Einstien, I would have agreed (of cource, not giving the interfering team any title). Since Jon Dudas's letter clearly states none of these teams were involved, I think FIRST made the right decision.

[techhelpbb]

Originally Posted by nukemknight

I know the report is long, but I urge everyone who wants to make a statement about the report read the entire report first. There have already been multiple posts in this thread containing false assumptions.

I saw this already as I've read it three times. I consider my statement correct.

They assumed that the AirTight system was going to trap such attempts and they assumed wrongly.

The point remains that it's highly likely that there are many other things that AirTight won't trap (2 other that I am acutely aware of) that are not already discussed in this report.

In short, yes you can read this that I limit my concerns to deauth but frankly the solution isn't to fix one problem in AirTight and think that it's now fool proof security. That's the sort of thinking that created the problem.

They need to completely reconsider how they transit the really important traffic. Custom solutions in their context could mean anything (including loading existing hardware with DD-WRT or OpenWRT which itself is not free of exploit).

The long term risk is that all the focus sits on this particular vector of attack and polite offloading of all security concerns to AirTight continues to leave exposed other vectors of attack. It's not about blame at this point. FIRST has gotten more blame than they deserve in some ways and AirTight doesn't make robot WiFi security products specifically for FIRST. It is just about suggesting that AirTight has only that issue which is wrong and is what this report basically does. It's not the be-all-and-end-all of security solutions (almost nothing ever is regardless of what sales says).

[Al Skierkiewicz]

Everyone,
Now that the report is out, I urge you to read it in it's entirety, twice. Then sleep on it. Please do not cherry pick from specific parts of the report and draw conclusions about the system, the robot or the hardware in general. Note that each team is handled separately so that you can understand specific issues that occurred during the matches on Einstein. Please do not generalize a statement from a specific team report to indicate this occurs for all robots. (e.g. The Crio reboot time for Team 233 only was found to be 24 seconds.) Also, I urge everyone to stop using the generic term "communication failure" to describe the Einstein issues. The report is detailed enough that using that term is not descriptive of what actually occurred.
I would like to personally thank everyone involved in the Einstein weekend investigation. Each person was committed to finding answers, sharing data, and coming to a conclusion that would be of help to all of us. This will be one of those high points I refer to when asked why I continue to participate in this organization. There is a great group of special people involved. Special thanks, of course, need to go out to Bill Miller and Frank Merrick and all of the First staff who worked so hard since St. Louis to insure we continue to have a quality competition. Thanks to Jon Dudas and the FIRST Board for supporting this investigation and their continued commitment to excellence.