Re: Team 548 Einstein Statement
 

The issue wasn't what you listed... the issue was the intentional interference with the game play. All the items you listed are something an individual can pursue, so long as they do so appropriately. Doing so during a match is not appropriate.

Re: Team 548 Einstein Statement
 

I also work with security and I agree.

Unfortunately the back story in this case seems to flow in a direction that you'd end up making the public report.

I and others I know have since submitted concerns and vulnerabilities to FIRST and frankly no one I know has received so much as a confirmation e-mail.

So what this will lead to is a pretty serious problem. FIRST has an investment in this control system for a while and that while definitely includes this upcoming year.

I know for a fact that these vulnerabilities remain and their mitigation procedure will not address them so long as the control system remains essentially as it is.

In 6 months if I publish my results publicly I can't with a straight face ever look at a hard to explain robot failure and not assume that I provided the core bit of knowledge that someone of less skill used to possibly cause that.

This is a very bad situation. It does not excuse the interloper at all. It may not have been apparent to the interloper they would face this additional level of inertia in handling the security issues.

There have been moments in my long involvement with FIRST that I felt I was utterly and sometimes quite wrongly ignored. Even that said I can think of a dozen ways in 1 minute that I can get my point across without using Einstein like that and compounding the existing issues with harm to every aspect of FIRST.

I appreciate curiosity but I appreciate the value of the scientific method to satisfy that curiosity. There was no careful control for this experiment and therefore it's not an experiment. What it really is a bunch of intelligent people chasing individual agendas not working *together* and in the process making the situation much worse.

Worse Einstein has become the distraction for who knows how many other possible interruptions that could have been caused accidentally or with intent. There's nothing in that report that closes that door, worse the lack of logs literally blows that door wide open.

Re: Team 548 Einstein Statement
 

Let's consider that.

The real fields are almost only available during competitions.

This leaves I suppose the initial practice matches before the actual competition venues.

One of the items I listed you could do quite utterly by mistake (I'm not saying this person didn't have intention to try it, I'm just saying we have no idea how many other people did that by mistake).

Re: Team 548 Einstein Statement
 

You could also approach the FTA and say, "I know you're busy, but could you leave the field up for a few minutes at the end of the day? I've got something that you need to know about." You could also try in the morning before matches.

Let's think about it this way: You have a practice day (well, if you aren't in the districts, you do--even then you have some practice time). Do it to your own team then, it doesn't affect anybody else then--just make sure your team knows you're doing it. Typically, there's about an hour before matches start on any given competition day (depending on opening ceremony start time in relation to pit opening time--don't try anything during the ceremony!). And there is often a couple hours at the end of the day, with the exception being the last day.

If you think that there is a problem with field vulnerability, or other system problems, Do Not Wait. Talk to the FTA during any of those "down" time periods--or ask in a shorter break, say between matches, if you can demonstrate the issue during them. If you are invited to demonstrate it, that's when you should do it--during lunch may also be an option. You can bet that if the vulnerability issue had been demonstrated to an FTA before Einstein, it would have been fixed or blocked before Einstein--it's one of those cases where "one guy knows, so we don't know how many others know".

Re: Team 548 Einstein Statement
 

I agree with this completely.

I disagree with this. The level of testing required to deal with the interloper's actions was/is really beyond what I believe is practical for field testing. Having now setup and broken down a field for this year's competition 2 times I can not see how sufficient time and resources would be available to scientifically and properly do anything more than trip over the solution.

Great if they trip over it. Not so great if they don't.

Additionally I can demonstrate additional issues right now. I know for a fact that several FIRST people know about them. Following only the reporting advice to e-mail the address on the report a person would literally be left in a vacuum. I have made it a point to make this harder to ignore because I expect that someone will do something about it. I'm growing ever more concerned that is not the case.

By September FIRST is hard at work generating the documents and written parameters for 2013 in their final form.
It's now August 21, 2012. So logistically when and where is this exploration going to get done?

Re: Team 548 Einstein Statement
 

I will open by sharing that I feel good about the way FIRST has conducted themselves throughout this process. I believe that FIRST and the volunteers who participated in the investigation have demonstrated FIRST's values of Gracious Professionalism and Coopertition.

FIRST has shown respect for all of the individuals involved and the FRC community in their transparency and communications of the process and outcomes. They have investigated, learned and put plans in place to correct and improve their hardware, systems and processes. They have maintained their integrity and sensitivity to the Einstein teams and the FRC community throughout the process.

What concerns me about some of the FRC community's response and the FIRST FRC Team 548 Einstein Statement is what it reveals about the FRC community's culture. I have read some comments in this thread suggesting that the interference of the Einstein matches was somehow excusable or justifiable. After reading the report, I come away with the sense that the document actually minimizes the egregiousness of the action.

Certainly folks may and should be forgiven for failures. However, that does not remove the consequences, nor does it restore trust.

GP means that we compete like crazy and at the same time play fair, maintain our integrity, while showing respect for our partners and opponents. I know that there have been times when I have not been a gracious professional. When I recognize it, I admit it, apologize, ask for forgiveness from the person I offended and resolve to do better. I see something like that in their statement and I hope that they do come out of this stronger and better.

But ... what does it say about our culture that this happened and that there are attempts to excuse, justify or minimize it? I would echo what someone said in a previous post, albeit perhaps in a different context. We still have a long way to go.

Re: Team 548 Einstein Statement
 

No one decides to bypass responsible disclosure (one method is mentioned earlier in Andrew's post) and takes it upon themselves to demonstrate vulnerabilities during competition matches again.

EDIT: whoops, there was a 6th page and at least two people already said relatively the same thing:o

Re: Team 548 Einstein Statement
 

Starting today it's been 30 days since I sent my first e-mail about this.
6 months is the end of January 2013.

If I follow through with the 6 month process as it stands now I'll be giving the next interloper the perfect window of opportunity for 2013 by publishing in late January. FIRST who might do nothing with the knowledge till then would have little time to react. Worse FIRST will have solidified all their purchases and shipped all the kits of parts.

Suffice it say I'm not thrilled with this. Worse even if I don't point it out then depending on a number of likely factors these exploits will be readily available to any interlopers that we don't know about if they've stumbled on them.

If that's not a house of cards I don't know what is.

So if I publish that information I risk FIRST responding by sanctioning me.
If I don't publish that information who knows if or when it'll get exploited.

For those who get the reference:
'The only way to win is not to play' and unfortunately I don't mean looking for security problem.

Re: Team 548 Einstein Statement
 

I think a lot of people want to believe FIRST is a utopia where everyone is good and would never do anything wrong simply because we are all participating in a great activity. As such, incidents where bad things happen can be trivialized because people will think "Oh, there must have been a misunderstanding here, so and so would never do anything to harm anyone", when in reality FIRST has bad apples just like any large community.

Re: Team 548 Einstein Statement
 

It doesn't have to be exactly 6 months. One might contact them and say "I will publish these findings on X date unless this is followed up with and another effective course of action is carried out". I don't think anyone here would be against one who did that, or support the powers that be for sanctioning such an individual. The point is that it is responsible disclosure.

Re: Team 548 Einstein Statement
 

I understand your point. However, the issue remains. FIRST, not just your robots, the entire contest is a problem too big for the time it's given.

August leaves 10 days.
September they build the documents and the rules.
October and November they setup the kits of parts.
December is anything that rolls over and of course countless holidays.
January, February and March is already too late.

So in reality I've disclosed them to FIRST now.
If I wait until after next season who knows what might happen.

If I levy that sort of consequence on FIRST what might they do?
Cause clearly other people have openly declared risk before that was not mitigated.

It's not just about shifting a few days. It's about the body politic.

Re: Team 548 Einstein Statement
 

You took the number 6 months entirely too seriously. I quite literally pulled that number out of thin air just to let people know that 2 weeks is NOT an appropriate period of time. Obviously publishing just before another round of competitions might not be good. But I was assuming that if a person is intelligent enough to discover the vulnerability and be wise enough to know how to go about exposing it they would have SOME common sense. I guess that's asking too much from people though.

Re: Team 548 Einstein Statement
 

I think the 548 statement was the right thing to do, they should be proud of what they did.

I would also like to point out that I see FIRST as a "sport". Back in high school I was on the varsity football team and there was some "cheating" going on there too. But I would like to say that I have seen more backstabbing in FRC than I did in football. People are people and that will never change, if you have a person who is willing to talk behind your back, then they will do it in FRC too. I had some team-mates who are my friends do this to me and it really hindered the way people see me, and still do to this day. But I am working hard to fix it still almost 3 years later.

Re: Team 548 Einstein Statement
 

Common sense is anything but. After all so many wish so many others had it.

This is a situation in which you have on one hand a vulnerability and a certain set of skills, resources and knowledge to outline it.

The other you have an organization pushed to the limits exposed to that vulnerability and perhaps not inclined to deal with it.

There's no reason...literally at all...to expect that I or any other researcher have the ability to influence FIRST corporate. That's the point.

The implied threat of exposure is a weak threat with FIRST because FIRST is a corporation with hundreds of thousands of kids impacted by it. You're not just costing their corporate bottom line or reputation. As all of these similar topic represent you're messing with the kids and it's not one step removed like disclosing some banking data.

Unfortunately this matters. There are too many disclosures I'm aware of and the costs on the other side of that big stick are too great.

Re: Team 548 Einstein Statement
 

In my experience the notion that FIRT doesn't listen to people is incorrect.

The notion that one is threatening FIRST with disclosure is incorrect as well. FIRST should want to fix this issue (if they aren't there are other issues that are completely irrelevant to the discussion) and by letting them know you plan on publishing the findings at a later date you are simply being courteous and giving them a chance to fix the issue before it becomes public. No threats implied at all.