Possible solution to Einstein problems.

[BornaE]

Thanks to FIRST, I have been working at Simrex corporation for the last 5 years. We manufacture radios and setup wireless networks anywhere from slow serial radios to 600Mbps full duplex wifi and wifi like links over long distances.

There are lot of ways of completely rejecting protocol level interference(plain RF interference needs more than a cellphone or a laptop).

1. Lock each AP to only accept one client.
2. Read the MAC address of team radios while programming the radios and restrict the connection only to that device.
3. Use non-standard frequency channels.(still 5.8GHz but with frequency shifter by 1MHz)
4. Assign equal timeslots to each robot providing very deterministic latency and guaranteed amount bandwidth for each robot.

These are all solutions that we have implemented in our products. Consumer devices very rarely have any of these features as they would become incompatible with other wifi devices.

[shawnz]

Well, the system "worked fine" except for the firmware bug introduced in Week 4. Other than what is now a known bug, and has since been fixed, points 1 and 2 are unnecessary. The Einstein report also detailed that point 4 would likely be implemented for next year.

I don't know about the legality of 3, but it is your field, so perhaps you could expand on it.

[efoote868]

Get a robot controller that reboots in under 3 seconds.

If something happens right now that the robot resets, a large segment of the match is shot. If the robot resets more than once, they might as well have not competed.

For those of you old enough to remember the PIC robot controllers, you'll remember that despite all it's limitations the PIC was basically instant on, instant connection to the FMS.

[BornaE]

Originally Posted by shawnz

Well, the system "worked fine" except for the firmware bug introduced in Week 4. Other than what is now a known bug, and has since been fixed, points 1 and 2 are unnecessary. The Einstein report also detailed that point 4 would likely be implemented for next year.

I don't know about the legality of 3, but it is your field, so perhaps you could expand on it.

point 3 is completely legal. The system operates within the appointed slot given by the FCC. however, the system would not use the standard channels defined by the IEEE.

4 cannot be implemented next year without replacing all the radios used in the field and robot. I hope this happens.

[BornaE]

Originally Posted by efoote868

Get a robot controller that reboots in under 3 seconds.

If something happens right now that the robot resets, a large segment of the match is shot. If the robot resets more than once, they might as well have not competed.

For those of you old enough to remember the PIC robot controllers, you'll remember that despite all it's limitations the PIC was basically instant on, instant connection to the FMS.

I partially Agree with you. however it is not realistic to get a high performance system that boots in 3 seconds.

better option would be to include a battery back up system that can run the radio and the robot controller if there are any interruptions in power. This would need a very small rechargeable battery and a some circuitry.

[shawnz]

Originally Posted by BornaE

point 3 is completely legal. The system operates within the appointed slot given by the FCC. however, the system would not use the standard channels defined by the IEEE.

Sounds reasonable.

Originally Posted by BornaE

4 cannot be implemented next year without replacing all the radios used in the field and robot. I hope this happens.

Why? The AP surely already has the functionality (it's quite sophisticated AFAIK), and the robot radios don't share networks with each other. What would be the requirement?

Originally Posted by efoote868

Get a robot controller that reboots in under 3 seconds.

I don't want to start another NI/IFI debate (disclaimer: i'm on the NI side), but the cRIO boot time surely could be improved at least a bit, even with current hardware. I mean, vxworks isn't *that* heavy. I've seen PCs boot faster than our cRIO.

[BornaE]

Originally Posted by shawnz

Why? The AP surely already has the functionality (it's quite sophisticated AFAIK), and the robot radios don't share networks with each other. What would be the requirement?

The AP does not that have that functionality as it is meant to be used to provide Wifi connectivity to other standard wifi devices. The features I mentioned are not included in the WIFI(802.11a/g/n) standard thus each manufacturers implementations are different and not compatible with each other. This goes for both 3 and 4.

[ratdude747]

1. Better quality wiring inspections. A few of wiring errors mentioned in the report, be the cause of the problem or not, should have been easily spotted. I am not sure whether using the regulated port for the radio is in the rules or not (I'll check later), but that error in particular sounds like it could have been spotted during an inspection just by a glance at the PD board and what, if anything was connected to the regulated port.

2. When possible, a move to a licensed radio band. In terms of implementation, I suggest:

1. replace the current bridge with a 5 port Switch.
2. For non-field usage, one of the 5 ports is connected to some sort of an wireless AP or a wired tether.
3. For on the field usage, the port ^ is connected to the proprietary radio, which comes with the field and is installed on the robot during match queuing and removed after the match. This would involve 18 radios (3 per driver station) to allow smooth field operation.

[Richard Wallace]

Originally Posted by ratdude747

1. Better quality wiring inspections. A few of wiring errors mentioned in the report, be the cause of the problem or not, should have been easily spotted. I am not sure whether using the regulated port for the radio is in the rules or not (I'll check later), but that error in particular sounds like it could have been spotted during an inspection just by a glance at the PD board and what, if anything was connected to the regulated port.

See <R42.B>. Also see the fourth item under "Electrical" in the 2012 FRC Inspection Checklist.

Most important, see the Team Compliance Statement just above the signature lines at the end of the checklist: (emphasis added)

We, the Team Mentor and Team Captain, attest by our signing below, that our team’s robot was built after the 2012 Kickoff on January7, 2012 and in accordance with all of the 2012 FRC rules, including all Fabrication Schedule rules. We have conducted our own inspection and determined that our robot satisfies all of the 2012 FRC rules for robot design.

So yes, better quality wiring inspection. By the team, for the team.

[Lil' Lavery]

Originally Posted by shawnz

Well, the system "worked fine" except for the firmware bug introduced in Week 4.

Really? Because in the report, six communications failures were laid out that were not tied to the FCA issue that was created by that firmware update. And I seem to remember plenty of communications issues through-out the entire season, not just on Einstein.

[pfreivald]

Surround the field in a chain-link fence (or other Faraday cage). It's at the very least a good start.

[Greg McKaskle]

As in the other thread, could we be more precise than "six communications failures".

Three of those were caused by a loose sensor signal wire and code that spun when the value was out of range. The code loop was in auto code and didn't happen in tele mode. Thus tests in the heat of competition failed to repeat the issue.

One was a failed ethernet cable -- not sure if it was purchased or shop-made.

One involved wiring the radio to the unprotected 12V circuit rather than the dedicated and protected 12V.

The radio reboot due to wiring had to do with a suspended crimp connection whose movement weakened and broke the copper wire after repeated vibration and impact -- at least this is my understanding as to why the crimp failed.

Another issue mentioned in the report, but not in the root cause section was a faulty main breaker installed between Quals and Einstein.

Communications failure might be how someone would describe the symptom, but it is not how I'd describe most of these root causes when wearing my CSA hat.

I think it is highly valuable to discuss how to improve the system and identify the issues that are too difficult to debug, etc. Vague terminology and statistics based on small sample size will not help matters.

Greg McKaskle

[Lil' Lavery]

Greg, I was talking about the end result (symptom) rather than the root cause. Perhaps it was foolish of me not to go into further detail on the issue, assuming people had actually read the report themselves. But I stand by the assertion I made in the other thread that any system where the best of the best teams can have a 12.5% failure rate at the end of their season is a system that has significant flaws.

[shawnz]

Originally Posted by Lil' Lavery

Greg, I was talking about the end result (symptom) rather than the root cause. Perhaps it was foolish of me not to go into further detail on the issue, assuming people had actually read the report themselves. But I stand by the assertion I made in the other thread that any system where the best of the best teams can have a 12.5% failure rate at the end of their season is a system that has significant flaws.

Fair enough, but what I was specifically quoting (BornaE's 1st and 2nd suggestions) would not mitigate anything except an attack on the wifi networks. MAC filtering is not going to cause people to wire their robot better. Besides that, the report clearly outlines the need for more substantial documentation, and that's *always* a good thing.

[Greg McKaskle]

I agree with much of what you are saying. It was highly disturbing to watch Einstein unfold. I watched three of the Division tournaments almost in their entirety and the energy and competition was phenomenal. It was quality IRST. Then a few hours later some of the same teams were involved in something entirely different. As the report shows, a number of factors were involved, and I expect measure will be taken to lessen the chances and severity of each of them. Additionally, items having nothing to do with Einstein were already being investigated and addressed, and due to the Einstein investigation, new items were discovered such as the SmartDashboard memory corruption.

I believe there are a number of other factors that contributed to the perception of what was taking place, but unfortunately my kids are wiggly and I don't have the time to write a coherent explanation. Maybe I'll get a chance later.

Greg McKaskle