Re: [FRC Blog] Einstein Report Released
 

I think you're forgetting advanced functions, speed and customization for advanced teams. It can't be so simple that advanced teams can't take their robots to the next level.

Re: [FRC Blog] Einstein Report Released
 

I disagree - the team (or whoever on the team is aware of the individual) should release the information and acknowledge that it was an individual, not a team effort. Imagine the reputation of the team if the information came out sometime in the future by another source - it would not look good. If the team came out it would be seen as a gracious step forward. I highly doubt the entire team was aware of the individual during einstein anyway.

Re: [FRC Blog] Einstein Report Released
 

The male pronoun is often used in cases of ambiguity. That you took the time to take issue with this incredibly trivial part of my post indicates that maybe we all need to step away from our keyboards for a bit.

Re: [FRC Blog] Einstein Report Released
 

Wouldnt matter. From then on that person would not be known for their specific name but as a member/former member of team _______.

Re: [FRC Blog] Einstein Report Released
 

If you honestly believe that you can draw parallels between not attempting the coop bridge with very very good teams and deliberately sabotaging the premier FRC matches, I don't even know what to say... You can tell that there is some animosity towards these outstanding teams if you went to a Canadian regional, but if you think that someone wants to ruin their Einstein appearance for some twisted payback, I think you're sadly mistaken.

Please don't make generic assumptions when you have absolutely no proof, only speculation on whom it may have been. You know what they say about assumptions...
---

I believe that FIRST staff, volunteers, and Einstein teams handled the situation exceptionally well given the circumstances. Lets not forget that 12 incredible teams went onto the Einstein field, and 12 incredible teams left it.

For this one person who deliberately tried to sabotage the event, there are literally hundreds of thousands of people who would condone in. Lets remember that one rogue person shouldn't be regarded as a significant change in FIRST culture.

Re: [FRC Blog] Einstein Report Released
 

Keep in mind that no key == an incorrect key, although I don't know if that applies in this specific attack. If it does, though, it increases the likeliness of it happening accidentally greatly.

Also note that it seems only some versions of the AP firmware have the bug, so this is not a long-standing issue.

AFAIK, this is basically what QoS is for, and the way they've described it in the report, it seems they're not using it yet (but will be next year).

As a side note, I'm also in the "stupidity rather than malice" camp, but from the looks of it, it wasn't just one person responsible. Either way, the actions taken were appropriate IMO -- including the provided anonymity.

Re: [FRC Blog] Einstein Report Released
 

Should the team then remain anonymous to avoid potential scrutiny? Again, what if the information was released by another source? Coming forward now would be the best way to avoid scrutiny at a greater scale.

Re: [FRC Blog] Einstein Report Released
 

To add to what Al suggested, if you don't understand something in the report, this forum is a great place to ask for further explanation. The above is a good example. It is nearly impossible for all items in the report to be understandable to all that read it.

Alan did a great job of answering the question, but I'll add that the FCA is not itself anything to be worried about. It could be due to a typo, a forgetful user, or an attempt at cracking the password. The access point with logging enabled may log the mac address of the FCA client.

As mentioned in a previous post, the FW bug means that teams who used encryption for build season, opened their programming laptop at the event and unknowingly attempted to connect to their own robot with their build-season password resulting in an FCA. This was investigated to determine if it contributed directly to issues on Einstein.

Greg McKaskle

Greg McKaskle

Re: [FRC Blog] Einstein Report Released
 

Also, the biggest crime against 1114 at GTR East was the ramming of the already balanced (1219/1114) coop bridge by 2185.

2185 was not at CMP.

Re: [FRC Blog] Einstein Report Released
 

If FIRST isnt the one to release that information after getting right from the team, if this person is even related to a FIRST team, then the source should be punished as much or even harsher than the original culprit. For someone else other than FIRST, the culprit, or the team(if there is one) they are only trying to stir up emotions by releasing that information.

Re: [FRC Blog] Einstein Report Released
 

Thank you FIRST for your transparency and releasing this extremely detailed documentation to us in the FIRST community. A huge thank you to those who worked extremely hard on creating this report and to those involved in the thorough investigation and testing!

With that being said, what is done is done. FIRST has dealt with said individual, and everyone is attacking him like wolves a sheep. While you do have reason to be angry (especially those who competed on Einstein) what is this solving? We can't determine one's intent and if I knew who was involved I would hold such a huge prejudice towards that team/individual without reason. I'm angry that this happened but at the same time robot hacking was bound to happen eventually and it has crossed my mind over the years (not with malicious intent) but I wondered when the day was when we would see it play out. It is a shame when, where, and to whom it was done but you can't change it. I'm glad FIRST is looking into solutions to fix these problems to eliminate threats in the future.

One thing that stuck out to me was the detailed explanations of electrical/programming issues experienced during the testing. I have a new respect for our electrical/programming teams as even the smallest issue of a loose wire can cause so many problems.

my $0.02

Re: [FRC Blog] Einstein Report Released
 

Truck Town Thunder, FIRST Team 68, would like to officially support FIRST in the results as well as applaud them for the way it was handled. Situations like this are unfortunate and it can be difficult determining the best solution to this type of problem. FIRST Team 68 supports FIRST in their decision.

Re: [FRC Blog] Einstein Report Released
 

I agree with Gregor.

Also, Please dont start abolishing FIRST in Canada over a few things. I personally Love 1114, and 2056. And I KNOW that a lot of people from other Canadian teams like them too. They should be looked up to. I talk to student from both their teams and they are extremely nice and always up to talking. Dont start saying there is a problem with FIRST in Canada, because you don't know what you're talking about. (This response isn't to Gregor, but the person he quoted in his original post)

THE END.

Re: [FRC Blog] Einstein Report Released
 

I think the best solution (to make everyone happy and minimize damages) would be for the team to come out and say it was someone from their team who acted unintelligently at Einstein, they apologize for everything, Gracious Professionalism is their greatest priority, they hope the FIRST community will find it in their hearts to forgive them, etc etc, The individual's actions do not represent the team, and most importantly the individual will not be named.

With that, the people "get" the culprit, the team apologizes and gets a cleared name (and even some respect from us for doing the right thing), and nobody can crucify the person at wrong because they don't know who it is.

Although, to be completely honest, I don't see why needing to know this information matters. If the team doesn't want to come out in the open, it's completely understandable, and it doesn't make us as people any better knowing who did what.

Re: [FRC Blog] Einstein Report Released
 

It wasn't trivial, just too obscure.

It appeared you were assuming something* that wasn't stated explicitly in the report. Other assumptions not supported by the report are largely responsible for the widely differing opinions concerning the appropriateness/adequacy of the actions taken by FIRST.


*At the time of posting, I hadn't noticed that in a previous post you had used "s/he" and so you were obviously aware that the report did not specify gender. So I stand corrected.

Re: [FRC Blog] Einstein Report Released
 

I'm not entirely sure about that incident, but I had a sneaking suspicion even before Worlds that we were somewhat vulnerable to dropouts when being hit near the battery. It could have been that, considering it coincided with a collision right there with the corner of another robot.

Plus, it was only once, and the NI experts in the pits actually reviewed our code and found it to be fully functional and efficient enough (~50% CPU usage average).

Re: [FRC Blog] Einstein Report Released
 

I'm glad you brought this up. While there is a need to support advanced functions, there is also a limit to how far you can go reasonably.

How many teams actually use the second digital sidecar? What about the second solenoid board? How many use a solenoid board at all? Could you replace the solenoid board with Spike relays and be just fine? If you were to remove support for the second sidecar entirely, how many teams would actually be affected, and would it be an issue which they could not possibly work around?

What about the software end? How many teams actually use the DMA functions in the FPGA? What about the vision processing - Do we really need to do this on the cRio? What percentage of teams found that to be a good solution this year?

Think of all of the teams that did it the "easy way", and used the default dashboard to see the camera image, then added tape to their computer screen to indicate where the target should be. They outnumber the teams that actually did the processing on the cRio, since it was so hard to do reliably without disrupting other systems.

In my programming experience, an experienced control systems programmer can do advanced things with very little framework, given a few key attributes exist:
-You can rely on a deterministic execution timer - It is acceptable to skip this rule if you have a reasonably accurate idea of the actual iteration time, and the execution time isn't too large (anything under 30ms should be fine for FRC applications).
-You can rely that the data given to you is valid OR you can validate this data in some way (if a missing/ejected cRio module can cause valid but old data to get to the user program, it is impossible to detect and thus a failure of this requirement)
-It is assumed that raw read/writes of the IO is available (e.g. must be able to sample all inputs at the execution rate, and set all outputs at the execution rate), as this will be used by inexperienced programmers as well.

Having worked in the "other competitive robotics league" of VRC and used both EasyC 4 and RobotC 3 (the Cortex versions, respectively), I was rarely faced with a limitation imposed by the (very limited) programming environments. In particular:
-RobotC did not have the timed task structure that EasyC had. I could take delta_t and use it to adjust the wait, and that was accurate enough (plus I could use delta_t to compensate for any execution time errors).
-EasyC failed to directly tell me the state of the field (e.g. Enabled/Disabled/Autonomous), as it had a funny way of calling and terminating functions. I found that this violated the third rule, and this was my primary cause for switching to RobotC.
-RobotC language did not allow pointers in any form. This was limiting as most of my math library relied on passing pointers to functions for operating, but I resolved it.

The cRio currently does this:
-In LabVIEW, an RT task meets the first requirement above while consuming an exorbitant amount of CPU resources in the process. This is fine, but the other inefficiencies of LabVIEW push the CPU load through the roof. A normal task fails the first requirement, but like RobotC, we know the timing so we can compensate (and it's generally acceptable, although the timing of LV is much more unstable in non-RT tasks than RobotC).
-The last time I checked, it fails the second requirement above if the module comes out of the cRio after boot, as the FPGA calls return the data that existed when the module was removed (in our case, it ejected itself from the cRio while crossing a bump). I do not know if this has been fixed lately, I haven't checked.
-It passes the third requirement.


This illusion of more functionality (and the larger processors to go along with it) often just adds to the complexity of the system on whole for marginal gains. While marginal gains are good, the marginal gains in performance seriously affect the usability to new teams, and, to me, that is worse than a very simple system which reliably performs the "easy task" and forces teams to do live within the limits of the system to perform the advance tasks. While I'm not saying the old IFI PBASIC system was good, the PIC based system solved most of the advanced functionality issues the PBASIC one had, and was reliable and it booted fast. An IFI system with a small ARM user processor like the Vex Cortex would be amazing

You should never compromise the base functionality in search of advanced features. Ever.

Re: [FRC Blog] Einstein Report Released
 

Thank you to everyone involved for the thorough investigation, and communication to the rest of us.

The report said this

But I read this


...I have no other comments to make at this time.

Re: [FRC Blog] Einstein Report Released
 

Unfortunately, while we on Chief Delphi like to maintain that one member of the team doesn't represent the entire team, it doesn't always work out like that, especially in cases of extreme circumstances. Now, you, or I, or anyone else on CD may believe the individual doesn't reflect the team, but there will always be people who believe otherwise. Any way you slice it (and I, personally, don't put much stock in this), people could look at it as "Well, this is the kind of people Team Such-and-Such has."

An individuals actions DOES, in fact, reflect the team they're associated with to some degree. Which is why I think it is best that the team and the person in question remain anonymous.

Re: [FRC Blog] Einstein Report Released
 

I am seriously stunned. I'm stunned that someone would do this especially at an event like FRC. Sounds like a personal vendetta. However, I applaud that FIRST is being so transparent and that they securing the problems for the off-season events.

I think we all are upset and stunned, but we need to learn from it and move on. Yes this is detestable behavior from someone especially someone associated with FIRST. However, we can't let this hiccup cause us all to fight among each other and become sinister with each other. We learned a lesson that FIRST isn't magically protected against interference based on the good will of the participants. There are and will be more bad apples. It's the double-edged sword of working with kids/mentors that are gifted with technology. Hopefully the problems are fixed and we don't ever have to face this issue again.

-Dustin Shadbolt

Re: [FRC Blog] Einstein Report Released
 

In case you interpreted that section in this way, the white paper isn't needed to document the system -- there are dozens of engineering requirements and development documents, but to publicly release a comprehensible description of the system so that all issues involving robots or field can be resolved more quickly, with less guesswork.

Greg McKaskle

Re: [FRC Blog] Einstein Report Released
 

As the report showed, there were certainly a lot of issues with communications, coming from a lot of sources. And those are just 12 of the best teams in the world. Team error likely supplied most of the connection troubles throughout the season, just as it was the root of many incidents on Einstein. I severely doubt this is a repeat offense from earlier in the season.


With that said, let's look at the big picture. Every coin has two sides, even this one. It's not even close to fair what this "individual" did to those teams, or even their opponents. It never will be. But trying to look backwards, like thinking of replaying those Einstein matches, wouldn't be fair to anybody. Instead, appreciate what Frank Merrick and the people in FIRST have done, making the very best out of a terrible situation, and look forward to the more promising future. This incident has spurred FIRST into looking intimately at how the FMS works, giving rise to the potential for positive change that can make the years to come better than today. As much as it should, nothing will change from the 2012 FRC World Championship. Celebrate all twelve teams who were crowned Division Champions, and have faith that 2013 and beyond will be the best FRC seasons of all.

And it appears that we will never know the motives, or even the intent, of the "individual", so let's not pile on or ask unreasonable questions of or about the individual or his/her team. Going through the rest of your life without the opportunity to be involved with FIRST is already a tall order for someone who probably loved FIRST and poured as much into it as all of you have, but made one horrible mistake to bring it crashing down.

Re: [FRC Blog] Einstein Report Released
 

I think Leeland's (and others') reasons for allowing the person(s) who interfered with the matches on Einstein to remain unnamed are spot on. Based on my past observations of how the Chief Delphi community has handled situations where someone behaves inappropriately, it is INEVITABLE that both the person(s) and their team(s) would be metaphorically crucified. The response could have unintended consequences to the person(s) emotional state. In addition, the team would likely be unable to recover from the blow to its reputation. This is because, in my experience, the Chief Delphi community DOES NOT separate the actions of an individual from the actions of the team. Often times, I see members state that "your words/actions reflect upon your team."

For this reason, I would understand if the person(s) who engaged in the interference declined to make their identity known.

Re: [FRC Blog] Einstein Report Released
 

On a different note, this is a brilliant example for teams of how to do Root Cause Analysis. It's also quite incredible to see what happens when you go over even the best robots with such a fine-toothed comb. Everyone can learn lessons from this document -- and even moreso when the FMS whitepaper is released! :)

Re: [FRC Blog] Einstein Report Released
 

Throughout the season, we saw a lot of posts and statements about "connection issues" at event and champs. Many people blamed FIRST through the entire season.

However, as this report shows, there are a LOT of issues that can affect a robot's ability to perform on the field, and many of them are caused (unknowingly) by the teams themselves. In the future, we should all keep in mind that these robots are very complicated machines, and there is rarely a single root cause for "connection issues".

FIRST did a great job with this report, and the sheer number of issues they discovered with the individual robots really shows how detailed they were in their investigation.

As for the individual who caused interference on Einstein... It's all been said at this point. The individual has been punished, and there isn't really anything else we can do about it. Part of Gracious Professionalism is not pointing fingers. When we find bugs at work, we don't ask "who wrote that section of code?" We ask "Who is the best, most knowledgeable person to work on fixing this bug?" As a community, lets move past the actions of the individual and show our support for the job FIRST has done.

Re: [FRC Blog] Einstein Report Released
 

I did exactly the same thing and spent a couple minutes thinking of just how awesome it would look to have towering Van de Graff generators over the old Manchester mill buildings... :D


This report is really awesome. Two thumbs up for FIRST because they really did their homework. Interesting a lot of the "Oh it can't possibly be team XXX" turned out to be not the case, and I think it will probably serve first well in the future as the community will probably not jump all over them at the first sign of trouble -- especially if high quality work like this can be expected in the future. It's also a side of engineering (failure analysis & technical report writing) that many FRC kids don't get to experience, so it's a interesting exercise from that point as well.

I'm just glad that the "hacker" is no longer in a position to ""inspire"" students.

Re: [FRC Blog] Einstein Report Released
 

It's is in my experience over 17 years extremely common for people to assume that the field can not be the source of a problem. Often they are correct but that only makes it more troubling when they are not.

No problem I do that every day literally. Only it's more than 60,000 people. I do computer security for a living as well as operating a few businesses that work with computing, electronics, and electrical. We have lots of security problems and we do our best to identify, qualify, quantify, document and offer resolution.

Sometimes we get solutions and sometimes we do not.
If we don't get resolution then we know where to look when the trouble starts.

It's off topic but that's a bad example. Anyone competent wouldn't store a password in clear text in a database with that sort of exposure to risk. You pad, hash and salt (and it's very simple there are existing tools to do this for you). They obviously left this old stuff laying around without regard for the SQL injection attack that is all the vogue for XSS these days. In point of fact we've been using this as a wonderful example of exactly why I have a policy document for the developers to avoid this exact attack vector (they are only very lucky that it wasn't a black hat that went after them quietly). They were also not very forthcoming about the possibility of the scope of the breach as they have a XSS single sign on they implement. Worse...some people think it was 'Yahoo Voice' that was breached but there's another Yahoo service 'Yahoo Voices' (that's right it's one letter off and the reporters who have to handle the announcement are not keeping it straight). It's an example of everything you don't do if you value your security or your business before, during and after a breach.

I grant you they have lots of other security issues at Yahoo right now that I am well aware of as are plenty of others. Surely they are not the only company that fails to be vigilant or gets utterly complacent. I'm sure someone figured they were saving a dollar (and maybe they did).

However, not all breaches are equal. The more people know about a problem the more silly you will feel when you get nailed for it. FIRST's deauth vector is not new, Hack-A-Day exposed this very publicly last year and other sites well before that. All that was required to breach this? Download code.

Perhaps the most effective hack is not a hack at all. Social engineering is the easiest and most effective hack because it hacks people. However, you don't differentiate you consider them all the same. Social engineering hacks are also why what you write next will not be nearly effective as you think:

Actually it doesn't matter what spectrum you use or how obscure. It's radio and it can be blocked cheaply and easily (though obviously illegally...but they gotta find you and prove it). As long as it's wireless denial of service will always be possible if you're willing to take enough risk as the attacker.

Obviously a band less frequently used will make it more obvious what you are doing. However once you commit to those frequencies without recourse they could hold you hostage long enough that the cost to continue will be extreme.

I don't actually disagree that they should move some of this from the bands where people accidentally could interfere with phones and such. I just don't think it matters as long as the field aspect is assured. So in that regard I think the field comm. specific stuff should be put somewhere and let anyone use WiFi for whatever they like. Let the users deal with the security issues, finding channels, and if you like give them a solution that ought to work in that regard but get out of the business of letting student written robot code interfere in field comms. The fields comm. stuff is generally unique to the competitions anyway, outside of the competitions WiFi is plenty workable.

Re: [FRC Blog] Einstein Report Released
 

The attacker's motives don't strike me as dark and mysterious. The report specifically states that the individual contacted FIRST and explained what they had done. If they hadn't, we would probably still be speculating. Were their actions in St. Louis wrong, foolish, and harmful? Yes. But I hesitate to say they were malicious.

Imagine this: You're sitting in the stands at your regional, watching a match, and just out of curiosity, you try to connect to the FMS from your phone. The phone asks for the WPA key. "Of course," you think to yourself and give up on this little experiment. But you punch in a random password anyway, and to your horror, the robot on the field suddenly stops.

You try to tell the FTA, but he brushes you off. Lots of robots die for lots of reasons, and there's absolutely no reason for a failed authentication attempt to take out a connection.

Now what?

From reading ChiefDelphi, it's clear to you that this is probably happening all over the country. FIRST HQ seems to be ignoring the control system issues and is blaming the teams instead, and it makes you angry. (Go back and read some threads from March - this sentiment abounds!) Championships are going to be a mess. It's almost certain that others are going to discover the issue, and they will probably use it in a malicious way to gain an advantage in the competition.

So, in anger and wannabe-heroism, you do the only thing you can think of to get FIRST to listen...

Was this the wrong way to address the issue? Absolutely. But given the openness, sincerity, and determination we've seen from FIRST in recent months, I wonder if the attacker succeeded.

This investigation uncovered a treasure trove of software bugs and electromechanical faults, and I hope we will all build better robots next year because of it. It showed that problems, both incredibly complex and completely stupid, will happen to all of us. Major thanks to FIRST for such a thorough investigation, and I'm looking forward to the improvements next season!

Re: [FRC Blog] Einstein Report Released
 

I'll just leave this here.
Anyway, this will be my last post on the matter. The intentions can never truly be known.

Re: [FRC Blog] Einstein Report Released
 

He was speculating as to how he / she discovered the bug - not how he / she executed it in St. Louis.

Re: [FRC Blog] Einstein Report Released
 

Pretty close to malice.

That situation is of course a hypothetical one, but we can't justify or glorify the hackers actions because of the "silver lining" this investigation was.

Re: [FRC Blog] Einstein Report Released
 

This is going to sound strange but scouting app?

After all even the Einstein teams will compete off season.

Several teams have applications like this in the Google Play store that anyone can download.

Re: [FRC Blog] Einstein Report Released
 

That have live match information? Not even the FIRST database has live match-by-match data. It couldnt be anything else other than the FMS system this person was looking at.

Re: [FRC Blog] Einstein Report Released
 

The information contained in the report and the information omitted from the report was selected for a reason. Attempting to read between the lines will only grant speculative conclusions. Wild speculation is exactly what this report was attempting to mitigate.

The "Root Cause Conclusions" table on page 20 of the report lists fifteen instances of command response failures. Six of those instances have nothing to do with the "Failed Client Authentication" issue. Only one of the fifteen instances is "confirmed" to be because of the "Failed Client Authentication" issue. Eight matches were ran on Einstein, each with six teams participating. That's 48 potential opportunities for command response failures. Setting aside the FCA issue, 12.5-14.6% (depending on 2056's root cause in SF2-1R) of the opportunities still manifested command response failures.

Regardless of whether or not the root cause was the fault of the field or the team, a system with a 12.5% failure rate among the elite participants at the end of the season is simply not acceptable. This report was not about the practice day at an early season regional, but the finals of the championship event. A vast majority of this report is not focused on the Failed Client Authentication issue, but the numerous other potential points of failure in the system both on the robot and FMS sides of the equation. I'm glad FIRST is taking steps to try and improve this system and remedy the issues highlighted in this report, and this is far more important to me than debating the motives and proper punishment of an individual.

Re: [FRC Blog] Einstein Report Released
 

Great report and worth waiting a few weeks for. It's easy to become cynical about the organization, but this is an example of FIRST doing things right.

Quick thoughts:

• Disappointed but not really surprised that there was intentional interference. It's been a huge risk and getting worse now that many participants are carrying at least one 801.11n capable device in their pocket.
  
• Surprised that even the top-tier teams had so many electrical and software problems. I've seen a variety of issues among the regular teams at regionals and this does all suggest that control system support is inadequate even now that we're at the end of it's 4th season.
  
• The overall programming model is not well understood, as evidenced by the multiple issues with *Continuous methods. Better documentation here will be very welcome.
  
• Similarly, two thumbs up for the promised FMS whitepaper.
  
• Unfortunately the "individual" will become known. Not that many people are on/next to Einstein during the finals, and this community is too curious/dogged for the information to remain hidden.

Re: [FRC Blog] Einstein Report Released
 

The problem with not naming the person/team is that the entire FIRST community is now guessing who it is...and probably coming up its a few ideas (some come immediately to mind for me). Is that a better alternative...to have people guessing forever? Or wait until next season when the person, who might be a popular person in FIRST suddenly isn't involved anymore?

Re: [FRC Blog] Einstein Report Released
 

To anyone still showing a bit of mercy for "the individual", the standard quote from Woodie came to mind (source):

The 2012 FRC manual leaves out the first and last paragraphs (note to Frank for next year...), but to me it forms the most basic code of FIRST's programs. I'd have a hard time buying an argument that the individual's actions are compatible with what's above.

Re: [FRC Blog] Einstein Report Released
 

Perhaps he was *entering* the live match data. This is how our team's scouting system works currently, and I know a number of teams use similar techniques.

I should hope nobody is suggesting it is! I think the argument here was whether the anonymity granted to the person was justified or not, rather than the blacklisting.

Re: [FRC Blog] Einstein Report Released
 

If you think the situation at GTRE was as simple as "not cooping with elite teams" then sorry, you need to get your head checked. The problem at GTRE was teams trying to influence the coop points of matches they were not involved in through conspiracy and, in some case involving rookies, bullying ("Nobody will like you if you balance with these teams"). Not cooping is a valid strategy. Trying to convince other teams not to coop is not. FIRST made this clear at the Week 3 events through a Team Update (IIRC), an emailed team notice, and Dean's own comments at the Montreal Regional.

Sidenote: I don't believe the events on Einstein are related to those at GTRE. The GTRE issues could have occurred anywhere where there are two elite teams that stand heads and shoulders above even the 3rd best robot. Imagine, for example, a Michigan where the only powerhouses are 469 and 67. Or a California with just 254 and 1717. These two teams pair up and dominate year after year. Of course this builds up some level of animosity. This isn't a Canada issue; Canada just so happens to be where this issue surfaced.

_____
As for 1114's official statement, I agree wholeheartedly. Mistakes were made, purposeful interference happened. Don't hide it. The great thing about 1114 asking for a public apology is that, if they were in the same situation, they would apologize in a heartbeat. Reputation can be rebuilt, but suspicion can't be dissolved without clear answers.

We are constantly told that the Einstein teams are the Best of the Best, not only because they have the greatest robots/strategic minds of that year, but because they are class acts. Apologizing admits that there may be something amiss in team culture, or that the individual was a bad apple but wasn't originally thought to be. Admittance is a very good first step in the right direction. I for one would applaud any team that came forward to admit that a member committed sabotage and also said they are working to fix the problem in their own team.

Keep the individual anonymous, and I'm ok with that. But it's the team's responsibility to make sure that EVERY team member "gets FIRST." I think it's a good thing that an individual's actions impact the reputation of a team. It forces a team to look at itself and intentionally weed out bad apples and turn them into glorious, shiny red ones with no worms. Asking for an apology for being unable to do this, in my opinion, is justified.

Re: [FRC Blog] Einstein Report Released
 

Any laptop with a wireless can would show the list of teams on the field since the team number is used as the SSID.

Re: [FRC Blog] Einstein Report Released
 

What concerns me the most is that someone who finds out who or thinks they did will think it is their job to play hero.

All it takes is someone to start writing letters or going after this person professionally outside of the parameters of what FIRST deems required.

FIRST is the biggest victim here. Not even the teams. There is a valid tangible dollar amount of damage done here. Beyond FIRST these teams have no one to turn to ask for compensation. I really think it is FIRST who should decide how they dish out the punishment and we should respect that. If we do anything more I think you are starting to stray into a place where you put your personal concerns above FIRST.

I personally have seen what happens when just a few people decide they know better how something should work and will take it upon themselves to go that extra step to have things a way they prefer. It rarely ends up the way they thought because they might not speak for the majority who has a less personal investment in their more personal motives.

I can only point out that we as members of a community are dedicated to gracious professionalism and activity to support that which itself represents that ideal. If this person became known and a few people decided to go even a little too far we as a community would be tarnished but it would then be a self fulfilling cycle.

Let us move forward into the future. Let us additionally focus this extra outrage on the continuing risk that someone might know someone that might try to do this again. If you know that someone is specifically intending to interfere in match operations and has taken action to actually make that interference happen please tell someone. Much of the annoyance with this could have been mitigated if someone tapped this person on the shoulder sooner and discussed it. Security is everyone's responsibility. Not just AirTight or FIRST. To me it really is the same as any number of more common safety hazards that crop up all the time.

Re: [FRC Blog] Einstein Report Released
 

Okay, it's that time again where Libby chimes in with a seemingly unpopular opinion.
I want to stress that I am not trying to be accusatory, and I'm also not speaking for any of my teams, for FIRST, or for my family.
This is all me, and only me.

If you'd like to make a point, to an FTA, email someone at FIRST. Don't ruin the biggest event of the year and make a fool (or, you know, a criminal) out of yourself in the process.

I personally hope for the team to come forward. If this individual were acting alone, and the team can honestly say they didn't know about it, then the team should be able to say "Yes, Jimmy (or Susie) McHacker was a part of our team, we didn't know at the time what was going on, but our team didn't condone the behavior and they are no longer on the team because of their action. We're really sorry." (Obviously, being banned from the team is no longer necessary since they're barred from FIRST, but that's not the point.) Yes, there will still be people in this community that will reflect the actions of the individual onto the team... but those people would be wrong, and we'd know that.

^Exactly. It looks far better for the team to come forward now than have it disclosed at a later date by another team/individual... and knowing our community, that's going to happen at some point.

You're taking an incredible moral high road, and saying that this person feels sorry. Good for you for believing in people, but what if they don't?! Unfortunately I've seen plenty of attitudes that say "I'm GLAD Einstein screwed up, because FIRST sucks and they had it coming". Paraphrased from many emails/FB comments/tweets/what have you, but that's the sentiment.

I think you meant 'condemn' and not 'condone' but your point shines through. I sincerely hope this individual was acting alone and that this does not reflect a change in FIRST culture.

--

Regardless of the individual I want to thank the wonderful people at FIRST HQ, the volunteers that helped with the testing process, and, of course, the Einstein teams. This is the biggest disaster FIRST has ever seen and you all handled it with class and professionalism. Thank you for being shining examples of what FIRST teams and participants should be, even during the bad times.

I'll leave you with a Woodie quote...

Thank you, all, for helping us take care of it.

Re: [FRC Blog] Einstein Report Released
 

Wow, I just found this:
I personally applaud this bold move from Samsung. They've understood that we won't be able to move forward as a community until the individual Galaxy Nexus phone, and Samsung as a whole, is held to account.

Re: [FRC Blog] Einstein Report Released
 

Too subtle?

Re: [FRC Blog] Einstein Report Released
 

I just wanna say that I personally voiced concerns about several issues that did apparently impact Einstein. I was aware of deauth and what it could do. It's part of my daily responsibility to educate myself in security issues and I knew about that for more than a year...though I never imagined that someone would intentionally do that (assuming they did it intentionally which I think we may never know).

My daily activities involve risk analysis. The shear amount of risk means that more often than mitigate all those risks personally (which in a way I have done with by prototyping some things I've offered to FIRST) I spend time writing up that risk and making it clear to other people that they accept what I deem as those risks by not taking some mitigating action (whether it's the one I recommend or not is up to them).

I've been a vocal advocate that the risks for an outcome like Einstein in robot power quality have been present for too long (for years). That those risks having been under communicated or under addressed could be a real problem and this report somewhat vindicates that point. FIRST is taking the position that they'll educate but the core problem remains. We build robots that crash into things, are moved frequently while not under power and the same is true for the field. Things are going to break. It doesn't matter how much you write reports people need the tools to diagnose those issues within the time frame the competition offers.

I tried to offer FIRST assistance at Einstein via communications in this forum and later via communications up to and including requests in the official forum. As a majority the risks were accepted that's not my job to do a little dance of pride about that when what I worried about happened.

It is however part of the healing process to make it clear in the aftermath that we can't ignore the underlying process that accepted this risk and insure that in the future we all more fully acknowledge the risks going in.

There are tragic moments in my life where I have pointed out risks to people and a great number of people died including dozens of friends of mine because they took a risk I deemed as reckless and complacent. You can stand there in shock and worry about laying blame or use the failed responsibilities as a tool to honor that which was lost with practical goals in mind.

I just want to make it extremely clear. I personally get no joy from being right when something bad happens I may have warned about. It reminds me every day that people often set their priorities in ways that take risks and don't know what to do once the risk is proven with consequences.

FIRST has expended a great effort with this report. However, this is hardly the end of it. This demands that FIRST consider ways to make sure that power quality issues can be analyzed with in the time frames they desire to operate. It further demands they more actively consider the security risks to their communications systems moving forward in the grander sense beyond this one deauth issue. To do anything less is to ignore the lesson cause and effect is offering.

What isn't apparent from this report because it hyper focuses on Einstein is how much of this happened years before and how much of it happened into the seeding up to Einstein. The fact is it is entirely possible that the whole of the competition was shaped by deauth and power quality issues in no small way.

Re: [FRC Blog] Einstein Report Released
 

I assume you are making a joke, but I can't figure out what point you are trying to make.

Re: [FRC Blog] Einstein Report Released
 

Considering that Samsung just release the Samsung S3 I am holding in my hand forgive me if I doubt the validity of this statement in the sense it could be taken.

Such a move would financially utterly destroy their company and in point of fact cause massive financial damage to the manufacturers that support them.

Besides they are already banned from selling the Nexus:
http://www.businessweek.com/news/201...amsung-s-nexus

Re: [FRC Blog] Einstein Report Released
 

Do you have the source for this? Because to my understanding, Apple sued Samsung and told them to stop producing the Nexus.

Re: [FRC Blog] Einstein Report Released
 

Yeah, I guess. My fault.

All: Disregard the above, maybe I'll try again later.

Re: [FRC Blog] Einstein Report Released
 

I am constantly amazed at how sarcasm challenged many people are Nate. :)

Re: [FRC Blog] Einstein Report Released
 

It's a joke, guys. I think he was trying to make an analogy between incriminating individuals and incriminating a whole team. It's not a great analogy for a lot of reasons, though.

EDIT: Late

Re: [FRC Blog] Einstein Report Released
 

To be clear they can't stop the production of the Nexus or the Galaxy Tab 10.1. I can produce anything I like and so can they (assuming it's not something subject to military restriction). They just can't sell them in the American market.

I can in point of fact still buy the Nexus, the Galaxy Tab 10.1 (GameStop has them) and of course the Galaxy Tab2 10.1 (which I just returned the other day for specific technical reasons).

The court did not order them to even empty the distribution of the product. Merely to stop offering to sell new stock until the matter is resolved.

Adding:

Sorry but that's sort of not a funny joke. You do realize that I have relatives that supply parts to Samsung and I know other FIRST members that if they read that would be wondering where their paycheck will come from. It's a tough economy out there, and this is a popular topic for people to read. Please think before you spoof your joke might not be funny if someone else panics. Misunderstandings like that can themselves shift the value of a stock.

Re: [FRC Blog] Einstein Report Released
 

Maybe my moral road borders on "So high, the oxygen is too thin up there." So I understand what you mean. But for right now, it's "What if..." vs. "What if...". We don't know the person's motives. We don't know how they feel about their actions. Maybe they really do feel bad, maybe there is no remorse what-so-ever. We just don't know.

In my personal views, any way we slice it, all involved parties remaining anonymous just seems like the best route. Realistically, what does the person coming public do for the FIRST community and, possibly more importantly, the 12 Einstein teams. Obviously 1114 wants the team/individual to come forward, but what obligation does the team/individual have? Yes, it would be the "Bigger man" thing to come forward. But they don't need to come out publicly to apologize. I would recommend that the person in question write a personal letter to each of the 12 Einstein teams, FIRST, and MAYBE an anonymous letter addressing the FIRST community. I don't think we have any business knowing, personally, who the person or team is.

That's just my view. Maybe I'm being to forgiving or careless. Oh well.

Re: [FRC Blog] Einstein Report Released
 

The real risk is statistical. If we know the team by admission then the number of suspects for the individual is reduced to below 300 people. Then all you need to do is cull the list of former team members from the last year and you're down below 50 people who could have done the deed. That's actually a small enough number that a dogged investigator will find the person in question shortly and with specifics.

I wouldn't be surprised if FIRST has bound them legally to silence it would be the smart move.

Re: [FRC Blog] Einstein Report Released
 

My own personal wish is that this situation will serve as a somber discussion point within each team. What would the correct response be if YOU found this or a similar issue? What if ... 's are a very valuable teaching tool for life, and the FIRST community is a safe harbor for learning so many of those life tools.

The fact that this statistically unlikely combination of wifi components left the door to the FIRST field partially unlocked doesn't make it that unique. A lost wallet, an unlocked or easily bypassed physical door, an online account with a weak password, all of these take place day-to-day and are worth considering and discussing along with the Einstein vulnerability. All systems have weaknesses, and for me it is cultural expectations more than the strengths of the locks that provide a sense of security in my day to day life. I am not intending to make excuses for the field issues. Locks and alarms are certainly a necessary part of the solution, but what really counts is how people behave when the lock is missing and there are no witnesses.

Relatively speaking, I'm still new to FIRST, and I'm still amazingly proud of the impact it has on people. The level of trust and generosity displayed within the community is practically unparalleled. And yet, it will not maintain itself. It can be improved. I hope that the ultimate outcome of this unfortunate situation is that thousands of individuals consider their own actions in this and similar what if ... scenarios and use it as motivation for self-improvement.

Greg McKaskle

Re: [FRC Blog] Einstein Report Released
 

My mom's 3 day old Galaxy S3 randomly died this morning. This makes sense.

Anyway, I don't think the team nor the individual needs to come forward. The team could easily come out and say that they didn't know about it/weren't involved with it, but whose to say that's the truth? I could easily say right now, "I did it.", and if I phrased it as an eloquent apology, people would believe me.*

Also, just a random idea here, whose to say the team knows about it? Maybe the interferer didn't tell his/her team why they were leaaving, and just treated it as a retirement or something.

Just throwing out some possible scenarios here.


*For the record, no, that was not a confession, wasn't a smartphone owner when I was at CMP, and I was the only one on my team there.

Re: [FRC Blog] Einstein Report Released
 

Two things I noticed:

The letter from Jon Dudas, and the report itself state that the culprit of the intentional interference was not a member of one of the winning teams. If the team associated with this individual was not one of the 12 Einstein teams, why wouldn't Jon simply say that? It's possible Jon simply overlooked this, but the wording is interesting.

It also said that the culprit was fieldside. Trying to speak with the FTA during the confusion.

As I recall (I've only been to CMP once), the number of people allowed fieldside at Einstein would be restricted to the drive teams of those teams participating on Einstein, plus some volunteers.

That leaves a VERY small number indeed.

I definitely agree with others that the individual in question likely WILL be found out in relatively short order.

I also suspect that the 12 teams (or at least their representatives present at the investigation weekend) probably know, and were asked not to talk.

Re: [FRC Blog] Einstein Report Released
 

Do people at your job purposely hide bugs in other peoples code to get their rivals fired? I work as a software engineer too and take the exact same approach to bugs (they are unintentional mistakes after all). That analogy does not work at all here.

Look, there is more to this story. We actually do know exactly who did this and we know more about their motive than you'd think. No it wasn't curiousity or accidental. We had eye-witnesses to some suspicious behavior right on Einstein. We took pictures of the person. We told the FTA right on the spot. We even told our opponent 1114. We didn't put the whole puzzle together until New Hampshire (i.e. we didn't fully understand what we were seeing at the time) but now we know exactly what happened and who did it. This person was cheating plain and simple. It was definitely not a scouting app. Right now if we just wanted to crucify the team we could. That is not the purpose of us asking them to come forward.

Here is the reason we ask: We've already seen other specific teams mentioned and considered as potentially involved. I've also seen people minimizing what happened and even some saying FIRST didn't really figure anything out! Go ahead, how many of you have a suspect in mind! This is not a good situation to leave this community in. I honestly believed the team would have come forward by now to bring closure to this. It makes me extremely upset that they haven't because by their secrecy it damages the other teams that were there. We're trying to be *gracious* and *professional* by allowing them time to do the right thing rather than throwing the book at them.

For the people trying to rationalize or minimize this, you don't know the whole story yet. I believe that every case of interference in the report was in fact interference and if it comes to it we can present a very convincing argument why.

Re: [FRC Blog] Einstein Report Released
 

@Hjelstrom: As I understand it, being one of the people who was AT the investigation weekend, you may wish to be careful how much you say, especially when its things you don't have proof of.

Re: [FRC Blog] Einstein Report Released
 

Fair enough. I removed the speculation at the end.

Re: [FRC Blog] Einstein Report Released
 

Video here

RSL is clearly flashing while it's dead, suggesting the cRIO is still running. Symptoms match what the report says happens with FCA.

Also, 1717 in this video. Symptoms match again.

Re: [FRC Blog] Einstein Report Released
 

Those videos give me chills. Wow.

Re: [FRC Blog] Einstein Report Released
 

Wow, I'm completely shocked by what I read. I can't imagine what drove someone to do that.

I think FIRST handled this the best they could, though.

Re: [FRC Blog] Einstein Report Released
 

One question to ask when demanding the person to come forward and apologize is "Could I forgive this person?" If the answer is no than perhaps the only reason you want to know the name is so that you can string this person up. Which leads to more ungracious and unprofessional behavior.

Only if the answer is yes will there be anything productive coming from a public apology.

What the person did is terrible and there should be just punishment and consequences, but this person does not need to be forever hated or despised.

Re: [FRC Blog] Einstein Report Released
 

First off, I must thank FIRST for its thorough investigation. They did a pretty good job of keeping us in the loop when it was appropriate. Frank, Matt, Ryan, and Kevin were all very supportive through this difficult time for us.

To all the people who have thrown in their two cents, I ask you to consider this. You have absolutely no idea what we have been through this season.

I had to stand in front of a room full of my students, parents, and our sponsors and explain to them that after we had worked so hard, and come so far, that someone felt the need to intentionally disable our robot on Einstein and prevent us from even competing on the world’s biggest stage. How many of you can say the same thing?

I attended the Einstein Weekend with Eric Mech, one of our graduating seniors. Eric never left the school before 9pm this season. He was our lead programmer and poured his heart and soul into the team. All the teams on Einstein were brought together and told about the FCA attacks at a meeting on Sunday morning of the Einstein weekend. We were told that FIRST had evidence that our robot was specifically targeted and disabled. I raised my hand and asked what evidence they had, and then Frank Merrick told me that the person had openly admitted to it. I got up and had to leave the room. I was furious. Eric stayed, and handled the news a lot better than I did. He graduates this year not knowing of what we could have achieved.

Would we have won on Einstein? Maybe, maybe not. 180, 25, and 16 were a very strong and deserving alliance. If we had lost to them, or to 987, 233, and 207 on even terms, we would have held our heads high, and congratulated the better alliance. I'm proud to call many members of these teams friends. But the fact is we were denied the chance to even compete by one incompetent jerk.

Trips to Einstein don’t come around very often. Even if you build the best robot in the world, you still need a darn good bit of luck to make it there. I just hope we get the chance to go back and play on even terms.

We in no way blame the entire team for the actions of this individual, but do feel they should stand up and acknowledge that a member of their team was responsible for the FCA attacks on multiple Einstein teams, and potentially others at the Championship and other events. We will give them time to do this properly while being respectful to the innocent members of their team.

On a separate note, the events of GTR-E have absolutely nothing to do with the events of Einstein. All the Canadian teams at the championship were hugely supportive, and we thank them all for that. We are looking forward to playing with 781, 772, and 907 as well as the rest of The Eh Team at IRI next weekend.

I will close with one rather ironic story. I had a gentlemen come up to me on Thursday at The Championship, and thank me for a recent post I had made on Chief Delphi on an unrelated topic. He told me that 2056 is usually very quiet, and likes to let their robot speak for them. Under normal circumstances, this is the way we like to operate. Unfortunately, it’s difficult to let our robot do the talking when it sits dead on the field.

Re: [FRC Blog] Einstein Report Released
 

I don't think you understand what the actual problem was. The system is indeed vulnerable to a deauthentication flood, or even a fast trickle. However, there were no such attempts detected, and there is no evidence to suggest that any occurred. The testing did show that it was possible to disrupt the connection without triggering a warning, so the detection parameters need to be tweaked to something more appropriate to the FRC use case. Still, it doesn't look like this was something that actually happened during competition.

The confirmed problem was instead an unknown and unexpected bug in the access point firmware that broke the existing connection when another client tried to authenticate and failed. Nothing special needs to be downloaded in order to cause this bug to be expressed.

Re: [FRC Blog] Einstein Report Released
 

+1

Re: [FRC Blog] Einstein Report Released
 

While we're sharing dead robot videos, how about

-67 in MSC SF1-2 http://youtu.be/ZbTHqBdvgJc - Clearly dead for the remainder of the match, blinking RSL.

-67 in MSC F-2 - This to me shows symptoms that could also be described as a loose ethernet cable. Team 67 dies in the key, and recovers when they are hit by team 469 (~15secs dead based on video). They then die later in the match, near the opponents bridge, and do not recover. I will share it anyway. http://youtu.be/CDLzSMPyYsc
-We died as well in the SF's under similar circumstances. I will not share the video.

I (as a member of the drive team) was by the field during all of those matches. In the all cases, the FTA's blamed the issue on the radio, and team 67 replaced their radio with a spare after SF1-2. I talked with team 67 (and team 469, their partners) personally after/during this, and they very thoroughly looked over their machine and found nothing (to my knowledge).



As to all of my earlier rants on the control system, I have two points that I was trying to make:
-The system is too complicated or difficult to setup if so many of the highest level teams cannot get it right.
-The system, while vulnerable to an unknown bug, was also vulnerable to a known, relatively simple bug (deauth) and there was nothing in place to detect or log this properly.

Re: [FRC Blog] Einstein Report Released
 

Thank you for posting this. This personal account is just an example of how one person and one team was affected by this, and it's heartbreaking. The person responsible for this owes, at the very least, a personal apology to every single team that made it to Einstein this year. You not only made FIRST look bad, you took away something that in all likely-hood these students, mentors, teachers, parents & sponsors will never get back - a fair shot at winning the 2012 Championship.

I really wish the reason behind all the issues were purely related to technological failures, because we can fix those. This is a people issue. People are much harder to fix. I'm encouraged after reading the report that the technology-aspect of this competition will improve for next year's season. Let's strive to improve as people and as a community as well. No one should have to go through what these teams went through ever again.

Re: [FRC Blog] Einstein Report Released
 

What was done is unforgivable.

I do not believe everlasting hate is justified, but I see nothing wrong with the person being forever shunned.

Re: [FRC Blog] Einstein Report Released
 

I understand how strange it is. I only threw that out there because it matched symptoms we saw in the shop once (with the practice bot, granted) while running it with our '11 bot.

Our lead programming mentor reviewed our logs, and they showed that we never lost communications with anything, indicating we "should" have been working the whole time. Strange indeed.

Re: [FRC Blog] Einstein Report Released
 

For everyone who is saying that the team should come forwards:

Do you REALLY want that team's members to be known as "the team that ruined Einstein 2012" for years down the road? In their careers, even?

Whether or not that is your intent, that is what will happen. The team will gain that reputation, like it or not, if they come forwards. No amount of "This was an individual, not the team, we as a team don't go that route" and "This individual is no longer part of our team" will stop it. If it was one of your students or mentors, and you put forth your team as their former team, could you handle that stigma for years to come, in addition to any other mis-informed and unfair ones you may already carry or aquire in the future? Or would you or your sponsors simply cut the team?

If the individual were to announce himself or herself by issuing an apology, while leaving team identity out of it, that's another matter. In that case, one person gets the stigma--again, they have to deal with it. But again, certain curious ones may probe deeper and discover the team. See above paragraph.

If this act was truly the act of a lone person, operating without the consent or knowlege of the team, then the best way to keep the team from being tainted is to keep the person's identity--and that of their team--a secret known only to FRC staff, and as few of those as possible. If, on the other hand, the team was involved in some way, then there are other measures that FIRST can impose, which they have not.


With that said: The report, as a whole, shows a thorough investigation of the Einstein problems. It was never meant to investigate all the comm problems in FRC this year--such a study would take a full year. However, it does provide a path forwards to eliminating comm problems, and better diagnostics in the future. I like the report, both as a very well-written piece of technical writing and as an explanation of what happened on Einstein, and also as a guide to places to start looking for other issues that may be plaguing teams.

Re: [FRC Blog] Einstein Report Released
 

A tad late to the party here, but here's my $0.02:

First, I agree with FIRST in how they handled things. Well done.

Second, shame on whoever hacked the field. That is anything BUT what I was taught was proper FIRST behavior. What they did is unacceptable. As for their affiliated team, I think that whether they reveal themselves or not is their exclusive decision. If the team was truly not directly involved the incident, then I could honestly care less about who the team happens to be.

Third, After reading the individual team sections of the report, I can now say that this proves the not even the best teams are perfect and ANYBODY can can make a mistake. Lesson: every team should double check their work.

No further comment.

Re: [FRC Blog] Einstein Report Released
 

Ugggh. I can't agree any more with this post. Regardless of how much "gracious professionalism" people have, regardless of how kindhearted people in FIRST are, THIS IS WHAT WILL HAPPEN. Call me cynical, but this is just how people are.

The only circumstances under which a public apology should be posted are, in my opinion:

1) Completely anonymous, sent by individual to FIRST, then posted by FIRST.

2) Sent to the main contacts of the "attacked" teams, then posted by someone from one of those teams, anonymously.

3) Basically, anything that keeps the identity of the "attacker" completely anonymous.


Again, I would love to believe that the general FIRST community would be accepting of it, but no matter what, the team would be referred to as "the team that attacked Einstein"

Re: [FRC Blog] Einstein Report Released
 

Unless you have something more than what is in the presented information I think you assume that AirTight which we now know can not see this issue under the right circumstances is sufficient reason to assume it did not happen. I draw nothing from this report that indicates to me they have actual raw data to confirm that a deauth attack did not happen on Einstein or elsewhere and it clearly is a well known vector with tools that often allow setup to exploit that hole in AirTight. Also you can make this work even if AirTight can detect it when fixed.

The second problem. The one you have listed as confirmed may be far more practical to point at and say well they did it and it requires no special tools we confirmed it. I actually mentioned that aggressive attempts to gain WiFi access could lead to this pages back. So I have acknowledged it but I think that concern about this and finger pointing is sort of crazy. There's actually premade devices you can buy that will locate and crack WPA passwords. Anyone could have brought one and had it in their pocket. Anyone at any point in the competition could have tripped over this. In point of fact well before Einstein suggestions about the versions of AP hardware surfaced. There where options to deal with this including spare parts in dumb luck you may get another version. The assumption here is that this person targeted teams with a mind to carry out rigging. How could they be sure the teams in question would be effected? They certainly can't walk over and offer to swap the AP.

On the other hand having now personally setup and torn down a field twice and looked at how AirTight is used. I am positive that a person could easily disable a robot or robots at will. They will still be able to do that when AirTight is patched and the AP versions are upgraded. I view this confirmation as a way to distract from the larger issue.

Re: [FRC Blog] Einstein Report Released
 

@techhelpbb:

The majority of teams were running the affected firmwares for the robot radios. You wouldn't have to be sure. It would affect most teams, and its unlikely you would have ended up with a full alliance of teams that couldn't be targetted in this way.

Re: [FRC Blog] Einstein Report Released
 

First of all I'd like to say that FIRST has done a remarkable job in handling this problem. I can't think of another organization who could pull together this kind of report and would do as much work as they could to figure out the conclusions that they did. Bravo!

As for the interference, the individual in question got what they deserved and that's that. Whatever team they may have been a part of has no responsibility for them if they were acting by themselves. I personally believe that the team should stay anonymous because whether we like it or not, they will become "the team who ruined Einstein". Which of course they shouldn't, but unfortunately, that's just how it works.

As for the teams that were interfered with on Einstein, I hope all of you make a soon return to Einstein, because what happened is extremely unfair and selfish. For those of you that were effected by it, the FIRST community is here for you. All of these teams have shown the utmost professionalism, I don't know if there are many other teams thy could do so. While 180, 25, and 16 were a fantastic alliance and definitely have all the credentials of a championship alliance, I still feel sorry for certain teams who were unable to perform because of an outside attacker that they had no control over.

I have no doubt that us as a FIRST community will be able to get through this rough situation, and that the teams involved remain as successful as possible.

My $0.02

Re: [FRC Blog] Einstein Report Released
 

I can't agree with these posts more. At first I was stunned (still am), I think what happened is repulsive, I think an apology is owed, but it shouldn't be public. It should be anonymous to the 12 teams involved.

Some say that the team should step forward before it comes out later on. What if the team doesn't know a member on their team committed this act? Imagine if this was your team. Yes an apology is owed but if identities are made public that team and person will go down in FIRST history and not in a good way.

Re: [FRC Blog] Einstein Report Released
 

Just for the record: for the radio, it was a specific hardware revision that was vulnerable (Revision A), not the firmware.

Re: [FRC Blog] Einstein Report Released
 

While I would love to agree with EricH et al, I don't.

The people that make up FIRST are some of the smartest people on the planet. We can read between lines, look for evidence, and investigate. Many people already know who is responsible, and many more already know what team they were associated with. Lots of people will do their own investigation into what went on.

I recognize that some people will forever associate the team with the interference, HOWEVER, I think that with time (and not much more time at that), the information will come out from another source, and that will make the lasting impression much worse than if it is admitted to and apologized for by the team.

The report was released approximately 22 hours ago. Lots of people have already come to same conclusions I have.

Re: [FRC Blog] Einstein Report Released
 

Okay but the effected Cisco firmware was not present until week 4 per the report.
Also I personally have Team 25s A version AP they traded me in an off season event.
It had been acting strange on them so they swapped it at a venue before Championship.
It is on my workbench next to me right now.
I gave them a brand new unit in exchange at an off season event.

The thing is that this issue did not exist until week 4. Team 25 had a version B so we know that version was floating around.
Someone would have to have discovered this issue from week 4, assumed that no one would swap the radio like 25 did for the sake of it, and had to have a grudge against that one alliance....then decided to continuously interfere only to have some fortune that wiring and other issues work to their advantage?

Seems far fetched. More importantly Team 25 had issues with that A version before that venue. If the target is 25 why bother to do it this way?

Re: [FRC Blog] Einstein Report Released
 

Just for clarification, I've met some real dumb people involved in FIRST. Involvement in an organization that celebrates Gracious Professionalism doesn't mean a person follows those principles, and it certainly doesn't make someone smart.

Re: [FRC Blog] Einstein Report Released
 

To those conducting their own investigation: Leave the pitchforks and torches at home, please. Ditto for the tar and feathers.

What happens if the team is released now versus later?

Now, while emotions are running high, the team would probably end up on every blacklist in FRC. That means that nobody will want to deal with them. The team sponsors get wind of it, and possibly cut the team's funding. The team potentially folds due to the combination of lack of funds and stigma associated with being on everybody's "Don't pick them or accept them" list.

But what if you let it go for a year? Two years? Even four years?

Now we've got a new issue to deal with. A new game (and hopefully a flawless run in terms of comm issues) has left last year behind, and we've focused on a rule we don't like again. A different rule. A different issue. The team in question has had time to get some turnover--and to, if they choose, release that they were the team in question. The Einstein Incident has passed into memory--a painful memory, but one that can be looked back on without as much emotional turmoil. Two years, and there is more turnover. More memory loss/burying. By four years, there's probably been a complete student turnover, and possibly a large mentor turnover. Then you quietly release that such-and-such a team was at the root of some of the Einstein Incident--and people wonder what you're talking about. They go back, and go "Oh. That."


Now, I'm not saying that the team won't escape any consequences--I know enough about blacklists to know that some actions will put teams on them for years and years. As some of the top teams know who the offending team is, or have a reasonable guess, that team probably will run up against them--but not with them, due to blacklist--in eliminations at some point. But being on the blacklist of top teams versus being on the blacklist of every team that has one--well, if I was in that situation, I'd rather have somebody willing to play on my alliance in the eliminations.

Re: [FRC Blog] Einstein Report Released
 

Why are you focussing on 25? As you mentioned, 25 had version B at Championship (and so did 16). They were protected (whether they realized it or not) from the FCA vulnerability because of it.

Re: [FRC Blog] Einstein Report Released
 

Simply because Team 25 had a working A version router that kept having issues on fields prior to this. So they swapped it.

Why did their A version have issues like that before that then have them go away when the B version was added. Perhaps someone targeted them before that.

They told me that swapping that unit was all they had to do to fix their mysterious failures.

I specifically handed them a new AP out of my own pocket cost because I wanted to see if I could find something in there to account for the issue.

I did the same for another team as well.

Re: [FRC Blog] Einstein Report Released
 

Not true. I was there on Einstein with our alliance and at the testing. The only think I KNOW is that is was no one on my team - our phones were off. We remember the days when you would be tossed from an event if you were seen with a wireless device near the competition field.

Re: [FRC Blog] Einstein Report Released
 

And what if they don't? What's the proper course of action if they have no remorse?

Should we continue punishing them until they break down and say their wrong? They've already been blacklisted, now we should publicly out them? What if that doesn't work? Should we continue to "turn the screws" until we see them suffer?

While I haven't been to St. Louis, that's simply wasn't true in Atlanta. I was field side on Einstein myself in 2009, despite not being a volunteer or participant on an Einstein alliance. There are numerous people who find avenues to be field side at Championship.

According to this post, 1717's issues were caused by a bad cable on the robot.

Because he's on team 11 from Mt. Olive, New Jersey located about an hour from the home of Team 25. He's had personal interactions with team 25 on the issue at hand and knew of their usage of that firmware.

Re: [FRC Blog] Einstein Report Released
 

Also, your team (as well as 25 and 180) were specifically cleared of guiltiness :P


This. It's a shame for that team, and I'm sure the teams involved that have their reasonable guesses have harsh feelings towards that team, but there is absolutely no need for all of FIRST to be against that team as well.

Re: [FRC Blog] Einstein Report Released
 

I remember those days, too.

I was a field volunteer at CMP (Archimedes) this year, but my crew was not selected to work the field on Einstein. While walking toward the stands before the finals, I passed the field volunteer crew that had been selected -- they were getting a briefing from their VC, and one of the items she emphasized was "turn off the Wifi on your phone."

Re: [FRC Blog] Einstein Report Released
 

Several weeks ago we posted this response
We have thoroughly investigated our communication failures to the best of our abilities and at this time we still do not know the cause of the failures.

Re: [FRC Blog] Einstein Report Released
 

Several weeks ago we posted this response:

We have thoroughly investigated our communication failures to the best of our abilities and at this time we still do not know the cause of the failures.

Re: [FRC Blog] Einstein Report Released
 

I do not know the individual involved, or what his/her motives were. If I were to meet the person, I would say this.

I'm sure you are aware of the hurt, shock, dismay, and costs that your actions have caused our community. In FIRST, we are taught to be gracious professionals. Grace is defined as this:
I'm sure you have volunteered your free time for the betterment of students in the past, and for that I thank you. I extend the benefit of doubt that whatever the motives, there is a possibility you meant to do good and chose an extremely incorrect course of action for doing so, which I hope you now see was a mistake. I forgive you of your actions, and wish you the best to all of your future endeavors separate from our organization.

Sincerely,
Barry B

Re: [FRC Blog] Einstein Report Released
 

Since folks are attempting to identify FCA failures from videos and data from other events, let me share some diagnostic detail. In particular, a blinking RSL on a robot that isn't moving does not mean FCA.

If a robot radio never connects to the field at all, it is something else.
If it connects to the field but stops moving ...
Does the DS indicate that it is connected to the robot? The alliance wall light also indicates whether communication is successful.
If it shows battery voltage and other signs of communication, it is something else.
If there is no communication with the robot, it is time to determine if the radio and cRIO are on or off. The RSL doesn't convey much info, but if it is active, the cRIO is up and at least some of the user code is active.

That leaves us with the radio. The LEDs on the front show whether it is in AP or bridge mode and whether it is bridged.

The odd symptoms that point towards FCA are a robot that can be pinged even though no communications succeed and the robot will either return too quickly to be a radio reboot or will not return at all.

The reason the report mentions cRIO reboot times was to try and identify definitively whether other failures could fit the symptoms. Unlike the radio, this is dependent on the team's code and needs to be measured for each robot.

Greg McKaskle

Re: [FRC Blog] Einstein Report Released
 

I think we should learn from this and make sure it doesnt happen again. FIRST has done a great job to handle this situation. FIRST asks us to be gracious and professional even to the people we dont like, if we do not act gracious then how are we any different from the individual? Everyone makes mistakes.

Re: [FRC Blog] Einstein Report Released
 

I told myself I'd wait 24 hours to digest the report before posting.

First off, I think FIRST did an awesome job with this report, and totally exceeded my expectations. This is a great demonstration of the problem solving process, and it was handled extraordinarily well. Kudos.

As for the communication issues that arose, words can not express my dismay that an individual would intentionally sabotage matches like that. I think the rest of the thread has already appropriately discussed this, and I'll avoid beating the metaphorical dead horse.

With that being said, I think FIRST appropriately handled the situations they were situations very well, and I support their decision to keep the guilty party(ies) anonymous. Like EricH said, revealing the team involved would cause a long-lived association between that team and the unfortunate events caused by one individual from that team. And that's not fair to the team - the majority of which weren't involved (and probably unaware). Sure, people will find out, but as a community, we should try and respect that team and not spread rumours or encourage "public hangings" of anybody.

I know that the FIRST community is composed of some of the best people, and I believe we'll get through it eventually. Let's try and keep cool heads and avoid deepening any wounds we've picked up.

Re: [FRC Blog] Einstein Report Released
 

As I read the report, I've been looking for two specific points of interest:

1. Why was the there a significant amount of failures for just the red alliance teams?

2. Special interest in how much vision tracking contributes to network traffic, as we (beta team)... were concerned of overwhelming traffic from this for teams that wanted to process vision via driver station.


I believe point 2 with the network capping addresses this fear for future games... but I would like pursue this with actual numbers (not here in CD obviously). In short I'd want to know if the mjpeg frames cause a significant load in the network traffic, as I want to consider using mp4 compression going forward. I think FIRST action items will address some of this... I'm looking forward to getting these numbers!


As for point 1... this is something I observed while watching the Archimedes elimination matches, and the report also reflects this for the Einstein matches. I've observed that red alliances usually have won (e.g. 90%) this year in our 2 regionals (I wonder how true that is for other regionals). There may be some correlation of this and the higher seed being on the red team. I noticed for Archimedes elimination, an overwhelming victory from the blue alliances.


I don't want to talk about the political stuff but just want to throw some general things out there... 1. Is there a scapegoat political agenda going on? 2. No one can keep a secret as all will be revealed to those who want to know about it... it is just a matter of time. Why? friends tell friends, and those friends tell friends... and well you get the picture... just like FB itself.

(P.S. I do not want to know... I'm an engineer not a politician)

Re: [FRC Blog] Einstein Report Released
 

This may have been caused by our Ethernet cable that connects to the radio loosening to the point that it was no longer transmitting data to the cRIO.

After Finals Match 1 on Newton we took the robot out to be examined. we noticed that the Ethernet cable was slightly loose, so we re-zip tied it to the radio. After that we ran fine in Finals Match 2. I'm not saying that this was 100% the reason why we lost control, but it does appear to be so, at least to me.

I'm leaving this here as a reminder to all teams, rookies to veterans, to double check EVERYTHING before going onto the field, you never know what could go wrong.

Re: [FRC Blog] Einstein Report Released
 

I agree with torihoelscher!

Re: [FRC Blog] Einstein Report Released
 

What do you mean by this question?

Re: [FRC Blog] Einstein Report Released
 

I just counted 71 red wins out of 150 matches for Archimedes... 47% red, 53% blue.

47R/95 for Alamo = 49.5% Red wins
31R/76 for Dallas = 40.8% Red wins

Disclaimer: I only counted once, so I may be off by a percent or two. Perhaps someone can crunch the numbers for all the regionals?