[FRC Blog] Einstein Report Released

[shawnz]

Originally Posted by Jon Jack

The issue isn't just trying to connect to the network, the issue is trying to connect to the network - while entering an incorrect WPA key

Keep in mind that no key == an incorrect key, although I don't know if that applies in this specific attack. If it does, though, it increases the likeliness of it happening accidentally greatly.

Also note that it seems only some versions of the AP firmware have the bug, so this is not a long-standing issue.

Originally Posted by Joe Johnson

Fifth, it seems to me that FIRST (and the FMS) has one implied contract with the teams: We will get X% of your data packets from your Operator Interface to your Robot and vice versa within Y msec.

AFAIK, this is basically what QoS is for, and the way they've described it in the report, it seems they're not using it yet (but will be next year).

As a side note, I'm also in the "stupidity rather than malice" camp, but from the looks of it, it wasn't just one person responsible. Either way, the actions taken were appropriate IMO -- including the provided anonymity.

[Gigakaiser]

Originally Posted by dodar

Wouldnt matter. From then on that person would not be known for their specific name but as a member/former member of team _______.

Should the team then remain anonymous to avoid potential scrutiny? Again, what if the information was released by another source? Coming forward now would be the best way to avoid scrutiny at a greater scale.

[Greg McKaskle]

Could someone summarize and explain the more detailed aspects of the Robot Testing and Failed Client Authentication Testing? Specifically, what intentional interference actually happened, how did it cause problems, and what are they planning to do to fix the issue?

To add to what Al suggested, if you don't understand something in the report, this forum is a great place to ask for further explanation. The above is a good example. It is nearly impossible for all items in the report to be understandable to all that read it.

Alan did a great job of answering the question, but I'll add that the FCA is not itself anything to be worried about. It could be due to a typo, a forgetful user, or an attempt at cracking the password. The access point with logging enabled may log the mac address of the FCA client.

As mentioned in a previous post, the FW bug means that teams who used encryption for build season, opened their programming laptop at the event and unknowingly attempted to connect to their own robot with their build-season password resulting in an FCA. This was investigated to determine if it contributed directly to issues on Einstein.

Greg McKaskle

Greg McKaskle

[Racer26]

Also, the biggest crime against 1114 at GTR East was the ramming of the already balanced (1219/1114) coop bridge by 2185.

2185 was not at CMP.

[dodar]

Originally Posted by Gigakaiser

Should the team then remain anonymous to avoid potential scrutiny? Again, what if the information was released by another source? Coming forward now would be the best way to avoid scrutiny at a greater scale.

If FIRST isnt the one to release that information after getting right from the team, if this person is even related to a FIRST team, then the source should be punished as much or even harsher than the original culprit. For someone else other than FIRST, the culprit, or the team(if there is one) they are only trying to stir up emotions by releasing that information.

[BrendanB]

Thank you FIRST for your transparency and releasing this extremely detailed documentation to us in the FIRST community. A huge thank you to those who worked extremely hard on creating this report and to those involved in the thorough investigation and testing!

With that being said, what is done is done. FIRST has dealt with said individual, and everyone is attacking him like wolves a sheep. While you do have reason to be angry (especially those who competed on Einstein) what is this solving? We can't determine one's intent and if I knew who was involved I would hold such a huge prejudice towards that team/individual without reason. I'm angry that this happened but at the same time robot hacking was bound to happen eventually and it has crossed my mind over the years (not with malicious intent) but I wondered when the day was when we would see it play out. It is a shame when, where, and to whom it was done but you can't change it. I'm glad FIRST is looking into solutions to fix these problems to eliminate threats in the future.

One thing that stuck out to me was the detailed explanations of electrical/programming issues experienced during the testing. I have a new respect for our electrical/programming teams as even the smallest issue of a loose wire can cause so many problems.

my $0.02

[aspiece]

Truck Town Thunder, FIRST Team 68, would like to officially support FIRST in the results as well as applaud them for the way it was handled. Situations like this are unfortunate and it can be difficult determining the best solution to this type of problem. FIRST Team 68 supports FIRST in their decision.

[akoscielski3]

Originally Posted by Gregor

If you honestly believe that you can draw parallels between not doing the coop bridge with very very good teams and deliberately sabotaging the premier FRC matches, I don't even know what to say... You can tell that there is some animosity towards these outstanding teams if you went to a Canadian regional, but if you think that someone wants to ruin their Einstein appearance for some twisted payback, I think you're sadly mistaken.

Please don't make generic assumptions when you have absolutely no proof, only speculation on whom it may have been. You know what they say about assumptions...
---

I believe that FIRST staff, volunteers, and Einstein teams handled the situation exceptionally well given the circumstances. Lets not forget that 12 incredible teams went onto the Einstein field, and 12 incredible teams left it.

For this one person who deliberately tried to sabotage the event, there are literally hundreds of thousands of people who would condone in. Lets remember that one rogue person shouldn't be regarded as a significant change in FIRST culture.

I agree with Gregor.

Also, Please dont start abolishing FIRST in Canada over a few things. I personally Love 1114, and 2056. And I KNOW that a lot of people from other Canadian teams like them too. They should be looked up to. I talk to student from both their teams and they are extremely nice and always up to talking. Dont start saying there is a problem with FIRST in Canada, because you don't know what you're talking about. (This response isn't to Gregor, but the person he quoted in his original post)

THE END.

Originally Posted by Gigakaiser

Should the team then remain anonymous to avoid potential scrutiny? Again, what if the information was released by another source? Coming forward now would be the best way to avoid scrutiny at a greater scale.

I think the best solution (to make everyone happy and minimize damages) would be for the team to come out and say it was someone from their team who acted unintelligently at Einstein, they apologize for everything, Gracious Professionalism is their greatest priority, they hope the FIRST community will find it in their hearts to forgive them, etc etc, The individual's actions do not represent the team, and most importantly the individual will not be named.

With that, the people "get" the culprit, the team apologizes and gets a cleared name (and even some respect from us for doing the right thing), and nobody can crucify the person at wrong because they don't know who it is.

Although, to be completely honest, I don't see why needing to know this information matters. If the team doesn't want to come out in the open, it's completely understandable, and it doesn't make us as people any better knowing who did what.

[Ether]

Originally Posted by Alexa Stott

The male pronoun is often used in cases of ambiguity. That you took the time to take issue with this incredibly trivial part of my post indicates that maybe we all need to step away from our keyboards for a bit.

It wasn't trivial, just too obscure.

It appeared you were assuming something* that wasn't stated explicitly in the report. Other assumptions not supported by the report are largely responsible for the widely differing opinions concerning the appropriateness/adequacy of the actions taken by FIRST.


*At the time of posting, I hadn't noticed that in a previous post you had used "s/he" and so you were obviously aware that the report did not specify gender. So I stand corrected.

[Ekcrbe]

Originally Posted by Radical Pi

I don't know about those two, but after those semis 68 had unexplained comms issues in the finals, and they do use labview

I'm not entirely sure about that incident, but I had a sneaking suspicion even before Worlds that we were somewhat vulnerable to dropouts when being hit near the battery. It could have been that, considering it coincided with a collision right there with the corner of another robot.

Plus, it was only once, and the NI experts in the pits actually reviewed our code and found it to be fully functional and efficient enough (~50% CPU usage average).

[apalrd]

Originally Posted by cgmv123

I think you're forgetting advanced functions, speed and customization for advanced teams. It can't be so simple that advanced teams can't take their robots to the next level.

I'm glad you brought this up. While there is a need to support advanced functions, there is also a limit to how far you can go reasonably.

How many teams actually use the second digital sidecar? What about the second solenoid board? How many use a solenoid board at all? Could you replace the solenoid board with Spike relays and be just fine? If you were to remove support for the second sidecar entirely, how many teams would actually be affected, and would it be an issue which they could not possibly work around?

What about the software end? How many teams actually use the DMA functions in the FPGA? What about the vision processing - Do we really need to do this on the cRio? What percentage of teams found that to be a good solution this year?

Think of all of the teams that did it the "easy way", and used the default dashboard to see the camera image, then added tape to their computer screen to indicate where the target should be. They outnumber the teams that actually did the processing on the cRio, since it was so hard to do reliably without disrupting other systems.

In my programming experience, an experienced control systems programmer can do advanced things with very little framework, given a few key attributes exist:
-You can rely on a deterministic execution timer - It is acceptable to skip this rule if you have a reasonably accurate idea of the actual iteration time, and the execution time isn't too large (anything under 30ms should be fine for FRC applications).
-You can rely that the data given to you is valid OR you can validate this data in some way (if a missing/ejected cRio module can cause valid but old data to get to the user program, it is impossible to detect and thus a failure of this requirement)
-It is assumed that raw read/writes of the IO is available (e.g. must be able to sample all inputs at the execution rate, and set all outputs at the execution rate), as this will be used by inexperienced programmers as well.

Having worked in the "other competitive robotics league" of VRC and used both EasyC 4 and RobotC 3 (the Cortex versions, respectively), I was rarely faced with a limitation imposed by the (very limited) programming environments. In particular:
-RobotC did not have the timed task structure that EasyC had. I could take delta_t and use it to adjust the wait, and that was accurate enough (plus I could use delta_t to compensate for any execution time errors).
-EasyC failed to directly tell me the state of the field (e.g. Enabled/Disabled/Autonomous), as it had a funny way of calling and terminating functions. I found that this violated the third rule, and this was my primary cause for switching to RobotC.
-RobotC language did not allow pointers in any form. This was limiting as most of my math library relied on passing pointers to functions for operating, but I resolved it.

The cRio currently does this:
-In LabVIEW, an RT task meets the first requirement above while consuming an exorbitant amount of CPU resources in the process. This is fine, but the other inefficiencies of LabVIEW push the CPU load through the roof. A normal task fails the first requirement, but like RobotC, we know the timing so we can compensate (and it's generally acceptable, although the timing of LV is much more unstable in non-RT tasks than RobotC).
-The last time I checked, it fails the second requirement above if the module comes out of the cRio after boot, as the FPGA calls return the data that existed when the module was removed (in our case, it ejected itself from the cRio while crossing a bump). I do not know if this has been fixed lately, I haven't checked.
-It passes the third requirement.


This illusion of more functionality (and the larger processors to go along with it) often just adds to the complexity of the system on whole for marginal gains. While marginal gains are good, the marginal gains in performance seriously affect the usability to new teams, and, to me, that is worse than a very simple system which reliably performs the "easy task" and forces teams to do live within the limits of the system to perform the advance tasks. While I'm not saying the old IFI PBASIC system was good, the PIC based system solved most of the advanced functionality issues the PBASIC one had, and was reliable and it booted fast. An IFI system with a small ARM user processor like the Vex Cortex would be amazing

You should never compromise the base functionality in search of advanced features. Ever.

[Barry Bonzack]

Thank you to everyone involved for the thorough investigation, and communication to the rest of us.

The report said this

LIGHTING TEST
One of the differences between the Einstein field and all four of the division fields was the lighting conditions. The purpose of this test was to investigate whether that difference could cause control or connection issues. Lighting was brought in to replicate the

But I read this

LIGHTNING TEST
One of the differences between the Einstein field and all four of the division fields was the lightning conditions. The purpose of this test was to investigate whether that difference could cause control or connection issues. Lightning was brought in to replicate the...

...I have no other comments to make at this time.

[LeelandS]

Originally Posted by SuperNerd256

The individual's actions do not represent the team, and most importantly the individual will not be named.

Unfortunately, while we on Chief Delphi like to maintain that one member of the team doesn't represent the entire team, it doesn't always work out like that, especially in cases of extreme circumstances. Now, you, or I, or anyone else on CD may believe the individual doesn't reflect the team, but there will always be people who believe otherwise. Any way you slice it (and I, personally, don't put much stock in this), people could look at it as "Well, this is the kind of people Team Such-and-Such has."

An individuals actions DOES, in fact, reflect the team they're associated with to some degree. Which is why I think it is best that the team and the person in question remain anonymous.

[Dustin Shadbolt]

I am seriously stunned. I'm stunned that someone would do this especially at an event like FRC. Sounds like a personal vendetta. However, I applaud that FIRST is being so transparent and that they securing the problems for the off-season events.

I think we all are upset and stunned, but we need to learn from it and move on. Yes this is detestable behavior from someone especially someone associated with FIRST. However, we can't let this hiccup cause us all to fight among each other and become sinister with each other. We learned a lesson that FIRST isn't magically protected against interference based on the good will of the participants. There are and will be more bad apples. It's the double-edged sword of working with kids/mentors that are gifted with technology. Hopefully the problems are fixed and we don't ever have to face this issue again.

-Dustin Shadbolt