SparkFlexes became unresponsive on a mixed CAN bus of Flexes and Maxes

[CJ_Hill]

This year we upgraded our swerve drive from eight sparkmaxes to four sparkmaxes for turning and now four vortexes for drive. We are using the post-recall flexes with newest firmware. Our swerve chassis drove just fine for the past two weeks, on a simple setup of a PDH, Rio, four maxes and four flexes. We didn’t change any of the drive code, and there are no CAN conflicts.
Over the weekend we started adding all of our arm and intake motors (on spark maxes) to the robot, and a strange thing happened. Two of the spark vortex motors driving the swerve modules stopped working. They just sat there with the cyan light solid, indicating brake mode but no communications. In diagnosing the CAN network we couldn’t see any problems, but the other two flexes then stopped working as well while we were trying to diagnose the issue.
We put the robot up on blocks and did the following:

• unplugged every CAN device and verified that we could talk to all of them via the rev client - all flexes and maxes report. The entire bus can be seen on the Rev client from any of the devices.
• double checked each device individually - all spark max driven subsystems work just fine - the swerve modules turn, the intake turns on, etc.
• went to factory defaults on the flexes, then when that didn’t help we booted them in recovery mode and reflashed. Still, the Flexes just sit there in brake mode

There have been several threads on CD with issues like this, but if anyone has any advice, I’d be happy to try it - I’ve seen some comments say that undocking and redocking their flexes fixed a similar problem, and another that noted that once more than a few flexes were on the same robot, they disabled themselves. It sounds like we are seeing something similar, but I can’t find any throughline that could clearly be linked to our problem and suggest a solution. Note - there are no CTRE devices on our robot, so I suspect the previous firmware fix for that particular problem does not help us.
I’m interested in hearing not just about others with problems, but if there are teams with several SparkFlex controllers living in harmony on their bot, and if they have any experience overcoming other issues with the SparkFlex.
Thanks!

[viggy96]

We had some issues initially, solved by updating the firmware on all our devices, along with checking all our CAN connections, to make sure they’re all solid. Our robot is pretty solid now CAN wise, haven’t had any issues with devices being unresponsive since.

[CJ_Hill]

Thanks for that. We figured out what had happened. For some reason, our config routine started failing when we were setting slot 0 and slot 1 right after one another. The flexes were not returning the kOk response, even though they were previously for the same code.
One or two of the spark flexes had lost their kFF settings, and redefaulted to zero. So whenever we thought we were telling them to move, they just put out zero, hence a solid cyan light.
After skipping the write of the second slot, all of the flexes were updating their config correctly again at the start of the robot code.
At the moment everything is running well again.

[viggy96]

I’ve found that you need to have retries in your code for basically every setting (kP, kI, kD, conversion factors etc), and also read the setting back to ensure that it’s been set correctly.