During Champs this year, we had a Spark MAX completely fail in the middle of a match (controlled our arm). No status lights and no way to connect to a diagnostic mode. The NEO motor it was controlling had no observable physical damage and cables seemed fine, so we replaced it directly with a brand new Spark MAX after flashing our settings and latest firmware onto it successfully.
Upon powering the robot on, this new Spark MAX also had completely died, with zero LED lights displayed and unresponsive to USB.
Unsure what happened, we decided to power off and install a second brand new Spark MAX, then plug in one at a time and power back on to test: first PDP power, then CAN, then motor encoder wire. All of these were fine. Finally we plugged in the 3 motor wires and powered back on (robot disabled), and it immediately killed that Spark MAX as well.
After killing 3 Spark MAX, I decided to try replacing the motor with a brand new one and a 4th Spark MAX. This worked.
Went to the Rev Robot Service Center to tell them what happened. They told us there is a possible failure mode that if the wires were internally heated and shorted, this can permanently damage the Spark MAX even if no power is flowing to the motor (other than from the encoder cable itself). Rev’s recommendation was to always replace both motor and Spark MAX when there is an observed issue with the controller, and to throw the motor away.
I took the motor apart completely to take a look at it, no signs of damage internally and the encoder board looked fine, but I guess it’s hard to tell when the wires wrap into the the center of the windings.
This was an unfortunate $350 incident on our robot
I’ve heard one failure that can cause this is a short between power and data wires on the NEO side. You can check for this with a multimeter - if there’s continuity between any of the power wires and any* of the wires on the data cable, I would assume that NEO could fry any Spark MAX you connect it to.
(*I’m not 100% sure if it’s any wire on the data cable, so before you go throwing out motors, I would check with a known-good NEO to verify that there is no continuity between any data+power wires. I wasn’t able to find specifics in REV’s documentation from a quick look, but it’s possible I missed something.)
We also experienced a similar issue at a regional this year, and found it an expensive and ultimately difficult to diagnose issue. CSA’s were incredibly gracious and helpful, but this issue ultimately was one that defined a regional for us.
The recommendation for replacing both a damaged ($90) controller and motor ($50) - is one that make sense, but is also frustrating for me as a coach on a team where stewardship of our resources is critical, because they are limited. It is particularly frustrating because - to my knowledge - everything we asked of the motor and controller were lower than specified maxima. While it’s easy enough to diagnose a short in the motor - it’s also something that certainly didn’t pass our minds the first time it happened. I
Between the propensity to damage with reversed polarity events in input and this, it makes me wary to encourage students to use the sparkmax in their designs. The issues with falcon 500s have also been discussed ad nauseam, and are very valid. The combination makes it clear to me there still isn’t a particularly reliable way to use motors of this type, and to make both fiscal and physical plans for them to fail - which is hard because supply isn’t a certain thing, and the cost can balloon quickly.
We had the same issue this year at Championships. We were having repeated issues with the NEO on our arm extension and the culprit was an internal short on the NEO side, which did end up significantly impacting our performance during Quals. The REV Service Center gave us the same suggestion: to replace both the NEO and the Spark together.
We observed continuity between the NEO power and one of the encoder data wires. As far as we can tell, the NEO short ends up shorting the 12V power lines to the 5V data lines which connect back to the Spark. Initially, this resulted in a Gate Driver Fault; We took the Spark apart and saw some reflow on the IC which controlled the MOSFETs. The next two Sparks we connected to the NEO just died completely, no signal LEDs, no USB connection, etc.
While replacing both with a fresh NEO and Spark did solve the issue, it does discourage us from using REV controllers and motors in the future if such an issue on one part can completely break the other.
But the other brushless motor in FRCFIRST Robotics Competition, the Falcon 500, comes with both parts in one (at a higher cost) and if either breaks, the entire thing has to get thrown out. It’s a big problem for brushless motors in FRCFIRST Robotics Competition no matter where you buy from.
I would like to see better guidance from REV on this—if one component breaks, replacing both at once will make it work, but there should be published procedures for testing the motor and controller separately to diagnose which part caused the failure and should be thrown out and which (if any) may be kept.
Any chance I could GET that NEO with the internal connection between the windings and the sensors? I’ve been working on a NEO/SparkMax testing board for a couple of years and I’d like to see if I can dream up a way to test for this failure mode.
Shorts in the windings are also VERY difficult to test for with normal test gear. The best way to identify them is to spin the motor (ideally with a drill). A good motor has very little drag, a shorted motor has a LOT of drag.
A shorted motor will show high current and will run slower than you expect it to due to going into current limit.
Here’s the current state of the art:
I have an idea to add something like “high/medium/low” current indication. And, now, I need to come up with a “short between windings and sensors” detection. That one needs thought, as I don’t want to go and create a problem while trying to FIND a problem. If anyone has ideas on a slick way to do that sensing, I’d love to hear it!
Unfortunately we gave the faulty NEOs to the REV Service Center in exchange for fresh ones. They were quite interested in looking at them.
We tested the motors using continuity on a multimeter. Buzzed from red/black/white to each of the encoder wires. There was also significant resistance trying to turn the motor by hand.
I think a tester board is a wonderful idea. Let me know if there’s any way we could contribute.
We have our NEO motor still, since Rev said they will only replace if it was “our last motor” and we had no spares. We did have a spare, so they did not want to exchange it.
Can I ask though — where is there 12V power flowing through the NEO when the robot is disabled? These subsequent failures occurred when the motor was not trying to run. It would make more sense if the MAX died when the robot and motor was enabled with 12V power running through the wires. Is there 12V power in one of the encoder wires themselves?
I’ve heard a number of teams saying that when in this case you should replace both the max and the neo because odds are you’ve damaged both. It happens more then you’d think. If you have the resources replace both to save yourself the headache and figure out the damage to the other components later ( in a world champs event ).
We were running Sparks with CIMs…we thought we had a bad CIM…it was the Spark max. Of course we found this out AFTER we tore one side apart and replaced the CIM. Our coder found that the Spark was bad after we ran a couple of matches after the replacement. We promptly unmarked the “bad” CIM.
We have since gone to NEOs…no real issues on our KoP base.
Update: I did a continuity check on the encoder wires, and there is indeed a short between the white encoder wire and the 3 motor power wires. Per Rev’s documentation, this is the analog temperature sensor, which I suppose has shorted to the motor windings
You can see the T wire (white) with a trace leading over to the right, to a via. There’s another via just to the right of it. I’m pretty sure that the temperature sensor is underneath the board, probably surface mounted and looking down at the windings. It might be raised above the board., however, it doesn’t seem to be a through hole mount. Unfortunately, I’m two states away from my shop so I can’t actually photograph the important bits
Not sure if that even seems like a reasonable trade-off for more accurate/ timely temperature measurements. Hopefully this is something they can adjust.
Would be good to know the distribution of these occurrences between NEO and NEO550, plus how often this occurs following smoking a motor, with a brand new motor, or just at random.
I haven’t seen any reports on NEO550s with sensor shorts. Keep in mind that 550s tend to -totally- fry. NEOs have a fun failure mode where some of the winding gets shorted out but they still run. Bigger wire is my guess as to why.
Also, the Neo550 may not have a buried sensor. I haven’t gone hunting yet
