possible solution to the static problem

I have to jump in here-

during build season our team had horrible issues with static frying the ethernet ports on our driver stations. We had hooked up the cRIO on an old drive base and used it for programming while the competition robot was in the shop. We have ALWAYS had static charge accumulation on our chassis from our large rubber wheels running on carpet to the point where we would drag a grounding strip below most of our machines (with a sign that says “if you can read this we are screwed”)
What we found was that even if we grounded the driver station to the nth degree if we tethered the robot to the station to dump code it would wipe out the ethernet port. We did this twice in the build season and Andymark graciously did a quick resupply of components for us.

Our solution to programming once we determined the problem was to dump our code remotely and we simply had no more problems with the driver station.

That is until NJ…

Obviously you cannot program remotely in the pits at the event. Therefore we were required to tether the robot. Almost immediately driver station issues began and we solved them by adding a switch in the line between the driver station and the robot. Doing that prevents the driver station destruction but we already have blasted one of the ports in the 5 port switch.

During calibration of the camera on the field we also had a driver station power supply blow out for no apparent reason and only a trip to Radio Shack and $30 was able to fix it. (there aren’t resupplies in the pit area or on the Andymark page!)

Those at NJ may have noticed that our pit is fanatical about static charge now. We wear Static Guard like cologne. Nobody touches the robot or controls without a ground discharge procedure. But there are times when connection between the robot and the driver station is necessary and at those times we keep our fingers crossed and a new driver station ready.

But perhaps what is the most frustrating of all is the response we keep getting from FIRST - " OH we haven’t heard of any problems" . I’m sorry but I hear a roar of problems when I talk to teams and I am not making this all up.

All we want is a control system with the reliability we have had in the past. I like the cRIO unit well enough. It is the peripheral parts that seem to be “less than satisfactory”. And it would have been nice to have as greater degree of testing and ready support for this system before were received it in the AS WAS state it went out in.

WC :cool:

Hey folks,

Not sure if all your static issues are wheels and airlocks; our (1391) experience at Jersey was something else altogether. Almost all of us are spinning some sort of insulator (rubber, plastic, urethane, etc.) around - quite often - another insulator (PVC or ABS rollers) and when they aren’t enough alike … whoops. We have created VanderGraf generators that send spike charges from the rollers into someplace or another that finds its way to where we don’t want it to be.

During practice rounds, we watched our robot run for 25 seconds, then stop as the CRIO rebooted, then run another 10 seconds, reboot, etc. This was early on and no one else had run into this, so we began from ground (no pun intended) up back in the pits. Curiously, static wasn’t considered early - we looked at current draws, 24V supply, voltage differentials, code, etc - anything that we thought might stop he processor cold. Eventually, at 7:45 in the evening, I disconnected everything, including motors, ancillary code, and hand manipulated our belt collector - bam! same behavior as all day. We could shut off the CRIO with me as the motive force.

By next morning we had: 1. rubberized our PVC rollers with DipIt spray paint (to make the roller more similar to our belt); 2. put a 3/8" aluminum rod across the frame that touched the roller (to dissipate charge rather than having it build up and spike), and 3. begun to spray our belt prior to each round with Static-Guard (yep, the grocery store solution.)

No more problems. Ends up the static spikes were traveling through PWM cable connections - can’t expect any processor to handle that, other than to reboot. We built the static generator, after all. All the ideas mentioned previously won’t deal with the delivery of an internal spike to the processor, so think through your observations carefully. I really don;t think frame charge is the issue - you did insulate your processor board after all, right? The processor was just doing what it was designed to do to avoid damage. The clue we finally had that it was PWM was that even when the camera was turned off, and we had disconnected the wires to the Victors (but not the PWMs), the camera’s servos were ‘twitching’ the camera for a couple spikes before the CRIO cut out.

Hope this helps.

At Manchester people were having issues with driver’s stations frying themselves when running in tethered mode on the practice field. At the end of the first day they had teams searching for spares. It just seems to me that while the new system is fancier and more advanced it’s not as durable as the old IFI system.

There are some of us in FIRST who do understand the importance of ESD control.

If you work on or near sensitive electronics hardware ESD controls are generally the rule. In my experience involving spacecraft hardware at NASA/JPL, there are very tight restrictions and mandatory training to avoid ESD issues. Dave’s “other cars on Mars” (Spirit and Opportunity) were built with great care to avoid ESD.

We have requirements for use of grounded equipment, wrist straps, conductive garments and work surfaces (including floors), ESD-safe chairs with chains that drag on the floor, minimum relative humidity (>30%), air ionizers, avoidance of static-producing materials (i.e. no paper, styrofoam, teflon, adhesive tape or other non-conducting items) within one-meter of critical hardware. We use bleed resistors (10-100 MOhm) between electronics chassis and power/return line to avoid static charge build-up.

One of the tricky issues with ESD is that it can cause “latent” damage. A piece of equipment might continue to work after a “zap”, but in reality some internal damage may have occurred. The next zap - even a tiny one, can cause a permanent failure.

I think we’re learning lessons this year in the importance of ESD control. Basically, we’ve got six van de Graaff generators running around on a good insulator with equipment that is likely more susceptible to ESD than what was used in the past. The fact that we’re seeing DS and robot resets attributed to ESD is frankly frightening to me. The guidelines from FIRST to deal with this threat needs some beefing up.

There’s another thread dealing with this matter as well: Field Static Solutions

That thread should be merged with this one. Those out there with real-world experience in ESD control should post recommendations for teams. FIRST needs to take the appropriate measures with the field (some of which are already in place, like the earth ground at the alliance stations).

All,

I’m not saying we don’t have a static problem. I’m suggesting that some people are jumping to conclusions here and what we need is some level thinking…

I found Steve’s post about his team’s roller system most illuminating…

I also agree with Gary and Tom that we are not getting the complaints of static from the robot operators and field reset crews that we did in 2004 ( http://www.chiefdelphi.com/forums/showthread.php?t=27912, http://www.chiefdelphi.com/forums/showthread.php?t=27914, et cetera).

Also, It would not appear that the floor and wheel material are far enough apart on the triboelectric series to be effective as static generators. Although I’d love to hear the input of a materials engineer on this…

On a slightly deviant note, it is interesting that we are reporting problems when tethered on the practice field. My team also runs untethered at the school… If this is true, I would think that this scenario (running tethered) should be reproducible in any of our classrooms when we are practicing.

I plan to look into this last point tomorrow…

Regards,

Mike

Glad to be illuminating; more often the kids on my team will tell you I generate heat instead. One more thing - we ran for two weekends in our gym untethered (and tethered in our shop) without ever seeing this issue. That’s why we were confounded about it rising in the regional. I think it ended up being a difference in atmospheric conditions (underneath the floor in Jersey is a solid-ice hockey rink) that made it far more dry than where we originally designed and ran our machine.

Just as an anecdote to this:

While we didn’t have any shutdowns or ‘no comm’ failures during our regional we did have 2 opponents robots shut down while we were in contact with them.

Our robot, with it’s propellers, generates a large static charge. Both robots that shut down had just intruded into our ball pickup section (we were in the act of pinning them … niether robots were near the airlock). Niether team ever figured out why they shut down … but I suspect static discharge. Our electronics system is completely isolated from our chassis.

Tim Baird mentioned getting shocked every time he unhitched a trailer at the end of a match.

To the best of my knowledge, I have never questioned that. If you can point to a specific statement where I have done so, please let me know and I will correct it. If you cannot, I trust you will correct your statement and your insinuation.

To the contrary, I am particularly concerned about making sure FIRST and the control system designers are able to gather accurate, factual, and correct information about the performance of (and where appropriate, lack thereof) the new control system. We are all intently interested in hearing first-hand, direct experiences from those that have had issues with the system, in as much specific detail as possible. Gathering as much of this sort of information as is available is the only way that the problems can be tracked down, diagnosed, and corrected. Posts like the one above from Steve Compton are full of great information and particularly useful. **

Conversely, unverified third-hand stories about how “I think my friend’s uncle’s cousin’s Little League coach’s team might have had a problem with a garstuckle that was ever so slightly out of plumb” are not particularly helpful. Frequently, they are actually detrimental to the process of getting the problem solved, as they cause a lot of energy to be expended chasing down false leads. So, it is important to identify and understand the difference.

-dave

** and toward that end, I would urge everyone with real, first-hand experiences with problems with the control system to please, please, please let the appropriate people know. Simply posting a story here on CD is not good enough. FIRST (in particular, Chris Jennings and Matt Pilotte) and the folks at National Instruments, Luminary Micro and KwikByte, have to know what is going on. They have to get real data on the performance, experiences, and problem reports associated with the control system. Please make sure you get your information and direct observations to them. One of the best ways to do this is to use the 2009 FRC Control Systems Forums.

.

I used to work at iRobot on Roomba, and some of the very early testing models had static issues (we were trying to cause them, we kept a special testing room at 10% humidity or less). As we insulated more and mor, and protected the processor more and more, the static would build up higher and higher until it eventually caused permanent damage when it did zap (it will always zap eventually). The adding static bleeders really helps the situation.

to the earlier comment that you might not be in contact with the carpet long enough before you hit the field wall - Electricity is pretty fast. If you have a good ground tail, it should bleed off the charge almost immediately.

I hope things are better this upcoming week. At least Seattle and Portland don’t have issues with low humidity.

When I was working at iRobot to troubleshoot this, we picked up a simple static meter to measure the static on objects. It’s very odd waving a box at your target and getting a static reading. The sensor on the front of the box was just a PCB with a giant copper pad that you had to point at your target. It was pretty cool.

Yes the FRP is non-conducting but the use of a few static discharge wicks in light contact with the FRP floor may equalize the potential between the floor and the robot. This is common practice in aircraft design where static discharge wicks attached to control surfaces and trailing edges help prevent buildup by discharging to the air. It is also good practice to bond any metal parts that may be floating from the main chassis, except of course for the camera and cRIO.

In Florida, we usually have such high humidity that static is rarely a problem. But this year, with the unusual cold, many days here remind me more of the static environment of the Northeast or Canada. Our students have been getting zapped left and right, much to their surprise. We haven’t had any resulting electrical problems (yet).

Depending on the exact mechanism involved in the static related resets, it could be that a static drain resistor of 100K - 1Meg between chassis ground and electrical ground might afford some protection. This would at least prevent very high static voltages from building up between the chassis ground and the electrical system without interfering with the safety aspects of the electrical isolation. In fact, it would be pretty easy to have a jumper with a resistor that could be installed (or not) to see if it makes a difference. Testing in a low humidity environment is certainly needed and perhaps this could be done in practice matches next week.

We had ESD problem a few decades ago with a communication device that output to line printers. The issue only cropped up in a few locations in the winter and IIRC we solved it with a drain wire on the paper basket and a couple of discharge wicks in the paper path. Although the technology was older (Z80’s), the problems were similar - mysterious unexplained resets.

I think FIRST need a “Tiger Team” on this to get control of the issue by next week. Just my $0.02

Perhaps, but as writchie notes, a grounding tail will allow static to discharge readily. We used some finely-stranded #10 wire, about 3 inches long, one end into a ring terminal and the other end fanned out into a good imitation of a paintbrush.

This was purely precautionary, as we have not experienced any issues where we might suspect ESD or static.

Ah, that explains that :smiley:

Folks should also note that just the spark of an ESD event is not the only chance for damage - the spark creates an electromagnetic field which can cause damaging voltages to be induced into near-by conductors. So it’s not enough to control when & where it sparks; you need to control the sparks themselves. Team 25, with their anti-static wrist straps and mats, are on the right track, This isn’t rocket science people.

Such latently damaged items are also called “walking wounded” and are especially problematic - the wound may also cause unintended operations, but it will be so intermittent it becomes difficult to find. And, in a high-reliability application (such as a spacecraft) it is a disaster.

I will ask our coach to ask Q&A to ‘approve’ a 1 MOhm resistor between components (frame, power, and cRio) for ESD bleed.

Don

Team-79 Antistatic solution
This should be tied to a bare-metal location on the robot. Allow about 2 inches of chain to drag on the floor.
As electrostatic charges are accumulated during motion, they are simultaneously drained from the robot. Thus dangerous levels never occur.

CHAIN BEADED 12" BRASS
SKU: 34733

77012 WESTINGHOUSE
$1.15

<http://www.acehardwareoutlet.com/(woa5vp55zc00kr55ubpfumrx)/SearchResults.aspx?SimpleSearchValue=chain+beaded>

Unfortunately, this solution would not appear to be permitted at present.

See the Q&A

http://forums.usfirst.org/showthread.php?t=12228

Hopefully there will be an officially sanctioned solution soon.

This problem has now directly affected us and our regional.

Both the ethernet ports on our DS were fried. The first occurred the first day of the regional. We began using the second port at that time.

The second occurred AS the drive team was connecting to the field controls for our first elimination match. We had just tested the robot tethered in the pits. The control panel went directly to the driver station - and they couldn’t get connection.

We were forced to remove the DS from our board and remove (12) PWM cables, fasten the new DS and reattach all 12 PWM cables in the time it took the robots to reboot. As a result, autonomous did not work. Two of the PWM cables did not make good connections. This allowed their human player to have their way with us. We lost by two points.

In the second match, the only thing that worked was the drivetrain joysticks. The turret joystick did not work, and none of the analog or digital inputs were sending signals. As a result we went the entire match without being able to score.

We will be grounding the robot from now forward AT ALL TIME if we are going to tether. Period. I strongly suggest other teams do the same.

We were told by field personnel that they had seen another (or more?) DS that had dead ethernet ports. Their guess was that the static discharge is being transferred down the ethernet cable from the robot while tethered.

Be careful and do everything you can to prevent it from happening to you.

actually this is a mechanical solution.
The rule that allows the chain is R05

<R05> Exterior or exposed surfaces on the ROBOT shall not present undue hazards to the team members, event staff or GAME PIECES.

We are not affecting traction as required by R06

<R06> ROBOTs must use ROVER WHEELS (as supplied in the 2009 Kit Of Parts and/or their equivalent as provided by the supplying vendor) to provide traction between the ROBOT and the ARENA…
No other forms of traction devices (wheels, tracks, legs, or other devices intended to provide traction) are permitted.

The chain is a mechanical device fastened to the robot frame and does not violate R41

<R41> All wiring and electrical devices, including all control system components, shall be electrically isolated from the ROBOT frame. The ROBOT frame must not be used to carry electrical current.

By adding a brass chain to the robot frame we have eliminated the buildup of electrical charges. This is a safe and legal method for protecting humans from electrical shock. It also protects the electrical components on the robot.

We experimented with a makeshift grounding cord made out of a shoelace that was attached to the frame of our bot. It dragged along the floor under the robot, to try to eliminate some of the static. It worked well… seeing in how we never fried our driver station.

It depends on where you are. Our CRIO as well as two other team’s were cooked over the course of the regional. FTA/ Field Coordination authorized all teams at the regional to use ground wires dragging on the field under the robot.

After everyone started putting these wires on, not a single CRIO failed.

The rules govern. The Q&A “interpretations” of the rules are official. The “lead inspector” is the final authority at each competition. That is how I understand it.

Re: <R05> this rule is prohibitive - not permissive. It doesn’t permit you to install something that would not otherwise be permitted.

Re: <R06> The lack of friction in the discharge “wire” was irrelevant to the Q&A answer. The question presumed the traction was immaterial.

Re: <R41> The Q&A questions was specifically in this context.

IMHO attempts to classify a brass chain, or wire, or metalized string, or whatever as “mechanical” and not “electrical” in order to escape an “electrical” rule are likely to carry zero weight. The actual use of the part makes it electrical, mechanical, pneumatic, or in this case electrostatic.

The referenced Q&A answer prohibits a bleed resistor between electrical ground and the mechanical frame and it prohibits a static wick between mechanical frame and the floor. The meaning of “no , and no” is pretty clear, even in a world where the meaning of “is” may be disputed. The GDC could have answered, yes and yes, no and yes, or yes and no, or even “maybe”. Instead they said “At this time, no and no, as these would be violations of Rule <R41>.” Since a “static discharge wick” is presently prohibited, IMHO the same would apply to drag chain.

I fully agree with you that a chain (or two) would prevent the electrical buildup. So would a static wick, a wire tail, a foil tail, and perhaps other methods. In fact, your chain idea MAY be the best. I’ve certainly seen it used successfully in the past. Perhap FIRST installing a drag chain on each trailer (which it controls and could easily supply) and allowing teams to install Static wicks made from off the shelf stranded wire will be the solution.

The GDC has determined that for the time being it wants to maintain floating electrical, mechanical, and surface grounds without electrical connection, although it left this open to later revision. While I am surprised by this ruling (especially regarding wicks), I am sympathetic as are most who have been in similar situations with mysterious ESD issues and a large number of units already in the field. At this stage, I think the GDC is trying to minimize the variables in order to reach the quickest possible solution. We should let them do their job. Perhaps, a couple of fried DS and cRIO’s are worth it if it quickly leads to a permanent solution. Maybe with week 2 now completed, they will promulgate a fix and/or rule change that is proven not to cause more harm than good.

Personally, I think there may be multiple problems. One results in a dead Ethernet port on the DS. The other results in a cRIO reset, and sometimes a fried cRIO. Since both are proprietary components, we don’t know what protection they have internally - we can only speculate. But since the DS doesn’t even have a pull up on the digital inputs, it is likely quite vulnerable. I doubt it has Ferrite Beads and TVS diodes on the Ethernet ports and it is possible that the actual damage comes from induced surges.

IMHO, the best course of action for teams is to have only ESD trained team members handle the Ethernet ports and insure that everything is discharged before plugging things in. I would also recommend that teams be prepared to install static wicks (or chains) if these are ruled as permitted. When operating tethered, I would recommend always using a hub or switch with known ESD protection and only connecting/unconnecting the DS cable in the pits.

As for good news, the forcast for the Florida Regional is 81F/61F with humidity of 40% - 50%. With the return of warm nights, the conditions for static should must less than they have recently been, although Murphy makes sure that ESD can get you anywhere. :wink:

Our team experienced this, though not at a regional. While I was away for other things, our driver and a few members were driving the robot on gymnastics mats. They quickly found me to inform me that there had been a huge static spark, and one of the jags had been fried.

Totally our fault.

In any case, to those talking about the old system being better- I am no expert, but I see no reason why ESD would not affect the old system (mostly victors). The actual controller would be safe, yes, but everything else would be just as at risk.