Re: Team 548 Einstein Statement
 

HUH?????

Should've clarified a bit. I'm not at all surprised FIRST didn't find it. There is no scenario in which they could've found the issue before Einstien. I'm saying that a) if they implement some sort of a feedback system, maybe the troubleshooting will be more comprehensive and b) the mystery mentor probably isn't the only guy who was aware of the problem before he tried to alert the FTAs. Just my 2¢.

Re: Team 548 Einstein Statement
 

uh oh. What'd I do, Al? :eek:


Oh. Yeah, then we totally agree with each other. (Does that make it 4¢?)

Re: Team 548 Einstein Statement
 

I am trying to figure what you are saying in that post.

Re: Team 548 Einstein Statement
 

Al, I think his point was that the likelihood of FIRST officials stumbling onto the FCA issue prior to Einstein was extremely small. The FIRST officials who would be most likely to recognize it for what it is (like the FTA's) are too busy during competition and matches to be flipping through their phones, so accidentally stumbling on it would be difficult for them.

I would add one more item... those individuals are probably the last ones who would actually "try" to connect to the field if the option pops up on their phone. They would just cancel out of the option and go about their business.

Re: Team 548 Einstein Statement
 

The assumption is a bit naive.

While I agree that 5Ghz wireless cards on battery-powered mission-critical laptops are far and few between (energy mongers...), any individual that tries to interfere from a driver's station laptop will probably not rely on a driver to do so. It's conceivable that the drive team wouldn't know it's happening. Most likely it'd go in a batch file or background script (rundll32.exe anyone?) that doesn't show up. Additionally, it could happen from the queue rather than on the field.

Now that an exploit is public knowledge, it's only a matter of creativity for how it's attempted to be abused. FIRST needs to find a solution for the root cause (sounds like they are). Turning wireless off for the laptops is a start.

Re: Team 548 Einstein Statement
 

It's hard to really enforce the zone around a field by just policing devices that are off.

You can't jam because if you do you probably will jam yourself unless you use a very well designed jamming system. Plus FIRST is a publicly visible corporation and you're taking your legal chances jamming like that. You can't count on the devices staying off after you look at them (if we assume no trust it's no problem to just turn it on or for an attacker to use resource kit tools to turn it back on). You can't even count on a spectrum analyzer and a near field antenna to find the devices because a device could be disabled when you look. You can't rely on denial of service detection because wireless by it's very nature is prone to short service disruptions which makes any channel disruptions less than a complete denial of service harder to detect. You can't even sort the process with a Bayesian filter because there are layers of complication and that requires some amount of repetition.

So in reality your choices to prevent future issues get quickly more difficult.

One could track communications losses per match and replay those that don't seem to be due to power issues to the radio (assuming we consider power issues to the radio to be a build quality issue). However, that does not fit with the current process that seems to be at work. Given the current process if an interloper can interfere and not get caught the match outcomes stand. So all it takes is someone with the knowledge and the willingness to absorb the risk.

Stick your head in on a DEFCON or Black Hat convention discussion some time. They'll pull stunts that obviously are pushing or breaking the law right in front of the authorities they know are watching them in the very same room. They aren't shy about it. It's going to be really hard to deny what they were doing if they get busted with a video of them doing it with an audience. At least they aren't concealing their efforts with what they know.

Re: Team 548 Einstein Statement
 

There are simply too many ways for a robot to fail (as we saw in the Einstein report) for the refs for FTA's to make a snap call to replay a match unless there is conclusive evidence that the cause was out of the teams control. When there is such evidence, matches are replayed. I've seen it happen when the field has had issues, which does occasionally happen. The issue is identifying the root cause of the failure.

So they're really already doing what you suggest in the last paragraph... it's just rare to be able to make the decision as to root cause.

Re: Team 548 Einstein Statement
 

I agree and with the logging on the field communications devices off and the robots mostly not logging the power to the radios (cause we were forbidden to do so per the official answer to my Q&A from 2012) there was no way anyone in the FIRST field crew would have had a good quick way to even narrow down on that issue.

Even if they monitor that radio power there are known and not well known programming pitfalls that can swamp the radios so I admit even the above wouldn't be entirely complete.

Quality of service monitoring on field side isn't a perfect solution either because the field channels can be swamped robot side or by the already mentioned wireless issues. One could track packet communications on the WiFi bridge as well but that'll probably require some custom firmware and some place to stick the data.

Re: Team 548 Einstein Statement
 

The specific exploit in question is no longer possible. The access point firmware bug that permitted it will not be present in the future.

Other exploits do still exist. Some are essentially impossible to prevent because they are inherent in the nature of 802.11 wireless networking and established security protocols, but they are detectable.

Re: Team 548 Einstein Statement
 

There is no way I can state my case that the remedies presented in the Einstein report will not be sufficient to prevent exploit in this FIRST related forum or any other FIRST forum publicly. If I make my case, eventually escalating to successful public proof of concept. All I'll be doing is enabling people with bad intentions. Proving my point is not worth the harm it will probably cause to hundreds of thousands of kids.

There is clearly no time remaining to do anything about the issues anyway.

Come what may. I'm glad that having the highest score is not my highest priority.

Re: Team 548 Einstein Statement
 

I seem to have done a pretty poor with this one. This is what I thought I was saying:

@Post 93 DampRobot questions how no one on the official FRC team could have known about this FCA hole.

@Post 95 I point out that it only occurs under limited circumstances, and say I'd therefore be surprised if someone from FIRST tripped on it themselves (while sharing the understanding that they would/should have acted had someone told them).

@Post 97 Astrokid points out that you don't need to know the cause of the issue to happen upon it, and says it's not surprising that someone just thought "What if I connect in during a match?"

@Post 105 (the "HUH?" one), I agree but draw a distinction--which, unbeknownst to me, Astrokid agrees with--between "someone" accidentally discovering it, and a FIRST official happening to do so. I draw this distinction because there are a lot more random "someones" than FIRST officials, FIRST officials tend to be rather preoccupied during matches, and given the otherwise extensive testing of the new Cisco firmware, FIRST officials have every reason to trust their selection and the system as a whole.

@Post 106, Al asks be what on Earth I'm talking about.



On a totally different note, does anyone know if the field saves the data records from the spectrum analyzer, or is it solely live feed?

Re: Team 548 Einstein Statement
 

Those with bad enough intentions will probably discover it sooner or later (or have already figured it out. Many exploits in software end up working this way). Disclosure is not always a problem. If you believe there is a reasonable mitigation (such as a firmware update, or more stringent procedures in pits+field) that could be made I'm sure many would appreciate it being public knowledge, especially if you have tried reaching out to FIRST already.

However, if you believe it is an issue with no easy mitigation that shakes the current technology foundation of the field and robot control systems to its core, disclosure might not be the best idea unless you are reasonably sure someone is using it.

Just my two cents.

Re: Team 548 Einstein Statement
 

I have both sorts of exploits and I have already disclosed this to FIRST 30 days ago so let's start with this:

For one the problem is the way the fields are laid out geometrically and the way areas of common play are positioned. I won't say why this is a problem I will say that a single WIPS sensor per field is not sufficient because of it.

There should be a minimum of 2 of those sensors per field diagonal from each other across the long dimension of the field. Take a good look at where the current AirTight sensor generally ends up and it's proximity to the Cisco hardware.

By the way, this was the very first thought to run through my head given the fact that one alliance or another seemed to be disproportionally likely to have issues.

Re: Team 548 Einstein Statement
 

Siri,
I read your post and thought that you were indicating that First engineering had already made the attempt to connect to robots by the time Einstein occurred. then I read further and became more and more confused as to what point you were trying to make. So let me make a few statements. No one at First, to my knowledge, had attempted to connect to a robot during competition. Of course they performed all kinds of testing in the off season and during other events. They constantly take in info from team members, even though they may not acknowledge that they received the info. They perform tests at HQ and ask FTAs to try things in the field. First also reaches out to trusted technical volunteers for their input and testing when needed, to insure that a good cross section of robot design and programming platforms are tested.
Teams attempting to control their own robots at home on their practice fields using 2.4 GHz wifi bands are common. I have done it with my phone and my robot. There are several apps available for Android and iPhone that identify available networks and some actually will do spectrum display showing network ID and signal strength. I checked my phone (I can turn wifi access on and off) between matches on Einstein and found a total of three at 2.4 GHz in addition to any robot radios that were on, and two of them were house Fan network points for use during games. Not what you would expect with all those phones and tablets out there in the stands. There was a spectrum analyzer in place to check for networks coming on line and searching for available connections. To my knowledge that does not keep a log but I can tell you several people were checking that during matches, myself included.
I would like to point people to the list of experts that were present during the Einstein weekend. In that list you will find people from Qualcom who were part of the design team for 802.11 communications and set the specifications, people from Cisco, RF experts from Deka, the wireless consultant that designs systems in the Boston area and worked on the FRC wifi design and a variety of First engineering staff, computer experts and RF Engineers. All of them brought or ordered up whatever tools they felt would be needed to analyze the field and robot communications. Their intention was to find what caused the failures on Einstein and to make an attempt to break the control system in use. I have not seen a group of people so anxious to break something and show off than those assembled. Yes, they found that there are some things that can be done to improve the wifi configuration and improve data transfers and prevent outages. However, and I can't stress this enough, during Einstein the only repeatable failure of wifi control that is supported by robot logs, observation, robot action, etc. was that of the admitted intrusion by a mentor on the field.