Spark Max Follower assignments getting missed

Technical
1

Hello SparkMax/Neo experts,

My team is using a chassis with 6 Neo/Spark Max controllers in a three left, three right configuration. We are assigning the 2 of 3 to follow the third as master for each side. While debugging adding current and rate limits to the groups I noticed a problem with the follower configuration. It seems that only one of the two followers would take the assignment. This was consistent for both sides

Code is simple – define the controllers

// CANSpark Max will be used 3 per side, 2 folowing the leader
    private final CANSparkMax frontRight = new CANSparkMax(FR_SPARKMAX_CANID, CANSparkMaxLowLevel.MotorType.kBrushless);
    private final CANSparkMax frontLeft = new CANSparkMax(FL_SPARKMAX_CANID, CANSparkMaxLowLevel.MotorType.kBrushless);
    private final CANSparkMax backRight = new CANSparkMax(BR_SPARKMAX_CANID, CANSparkMaxLowLevel.MotorType.kBrushless);
    private final CANSparkMax backLeft = new CANSparkMax(BL_SPARKMAX_CANID, CANSparkMaxLowLevel.MotorType.kBrushless);
    private final CANSparkMax middleRight = new CANSparkMax(MR_SPARKMAX_CANID, CANSparkMaxLowLevel.MotorType.kBrushless);
    private final CANSparkMax middleLeft = new CANSparkMax(ML_SPARKMAX_CANID, CANSparkMaxLowLevel.MotorType.kBrushless);

    private final CANSparkMax[] controllers = new CANSparkMax[] { frontRight, frontLeft, backRight, backLeft,
            middleRight, middleLeft };

Set the followers -

  // Have motors follow to use Differential Drive
        CANSparkMax rMaster = backRight;
        err=middleRight.follow(rMaster);
        err=frontRight.follow(rMaster);
        //err=backRight.follow(rMaster);
        err=middleRight.follow(rMaster);  //### hack - treat it like a teenager

        CANSparkMax lMaster = backLeft;
        err= middleLeft.follow(lMaster);
        err= frontLeft.follow(lMaster);
        //backLeft.follow(lMaster);
        err = middleLeft.follow(lMaster);   //### hack - tell them twice

I tried switching what was following what, it didn’t matter, only ever saw 2 of three lights work together. Even tried telling the missing controller a 2nd time… hack I know. That didn’t work either.

I suspect there is a timing problem because when I added the error capture and stepped through with the debugger, both sets of 3 where configured and ran correctly. That is, all the controllers showed Red/Green lights when commanded to move. Running code at full speed, only saw 2 of 3 lights again.

Stepping through with the debugger clearly changed the message timing to the devices.

This platform has 10 CAN devices: 6 spark max, PDP, talonSrx, RIO, and a PCM. All devices show up correctly on the CTR or SparkMax Client monitor tools. (I wish CTR and Rev would report other devices on each other’s tools).

When enabling current limits and/or rate limits I was sending the values to all 6 of the SparkMax controllers. See code below. Is this needed? Can I just send the updates to the lead controller? That would be the preference.

Do the followers simply listen for all commands as if they were on the master’s ID?

public double adjustAccelerationLimit(double deltaRate) {
        rateLimit = MathUtil.limit((rateLimit + deltaRate), 0.0, RATE_MAX_SECONDS);
    
        for (CANSparkMax c : controllers) {
            c.setOpenLoopRampRate(rateLimit);
        }
        SmartDashboard.putNumber("motorRate", rateLimit );
        return rateLimit;
    }

Given my assumption about CAN timing, this could be a problem. We intend to adjust the values in the real-time loop for tuning/trimming behavior.
When this code was run with a .5 second rateLimit the motor cluster started jerking and not moving well. Is this a known issue or any ideas what the cause could be?

Similar code was used to set the smartCurrentLimit and secondary current limits without any perceived problem, but we weren’t able to test the current limits were actually working yet.

The rate limit setting caused the whole drive train to stutter like the motors were fighting or simply dropping out.

Any suggestions would be great.
Full code: https://github.com/2202Programming/FRC-2020/tree/Larry
DriveTrain code: https://github.com/2202Programming/FRC-2020/blob/Larry/FRC-2020/src/main/java/frc/robot/subsystems/VelocityDifferentialDrive_Subsystem.java

Thanks for the help,
Derek

2

Haven’t had a chance to dig deep enough to figure out what’s going on, but I noticed it doesn’t appear you’re resetting your controllers. I always recommend that teams guarantee their controllers start from a known state. This is usually achieved by calling the “factoryDefault” method (restoreFactoryDefaults() in this case).

Also, please take a look at your driverstation logs and confirm you’re not overloading the CAN network (“CAN %”, rarely gets over 60% unless you’re doing something like fast-loop threading)

3

Thanks Fletch for the reply. They are getting reset, just didn’t include that code frag. Link at bottom for drive train will show it.

Update: I added 2mS sleep() calls between the follow commands and that cleaned up the assignments. It’s a hack, but it works. It is during a constructor so the delay doesn’t matter.

    void configureControllers() {
            // factory reset
            resetControllers();

            // Have motors follow to use Differential Drive
            CANSparkMax rMaster = backRight;
            err = middleRight.follow(rMaster);
            sleep(2);  //hack to ensure timing - 2mS
            err = frontRight.follow(rMaster);

            CANSparkMax lMaster = backLeft;
            err = middleLeft.follow(lMaster);
            sleep(2);  //hack to ensure timing 
            err = frontLeft.follow(lMaster);

            // velocity setup - using RPM speed controller
            this.leftController = new VelController(lMaster);
            this.rightController = new VelController(rMaster);
    }

    private void resetControllers() {
            for (CANSparkMax c : controllers) {
                c.restoreFactoryDefaults(false);
                sleep(2);
                //c.setIdleMode(IdleMode.kCoast);  //idle is default
            }
    }

We are still fighting smoothness in driving, there is a lot of power on this chassis and the default current limits of 80 amps is pretty stiff when the driver changes direction. Still working this out for what the right limits are.

Should I call restoreFactoryDefaults() so it burns the flash? My plan was to burnFlash() after things are working better at the end of the constructor.

Does anyone know if you can just update the masters once the followers are established?
I have a plan to test this, but haven’t had time due to other gremlins… for example rateLimits.

When I’ve added rateLimits the motors start to jerk and the driving is impossible. I am actually concerned about burning out the motors. I can’t tell if they are fighting because I’m getting partial updates or something more subtle. Here is the code I’m using to set open loop rate limits.
I’ve been using a fast ratelLimit of 0.5 seconds, but it causes lots of stuttering of the drive train.

/**
 * adjust the acceleration time - limit on how fast the output gets to max
 * 
 * @param deltaRate (seconds) amount to add to current rate
 * @return
 */
public double adjustAccelerationLimit(double deltaRate) {
	rateLimit = MathUtil.limit((rateLimit + deltaRate), 0.0, RATE_MAX_SECONDS);
	//TODO:figure out why this isn't working
	for (CANSparkMax c : controllers) {
		c.setOpenLoopRampRate(rateLimit);
		sleep(1);  //TODO: needed?
	}
	SmartDashboard.putNumber("motorRate", rateLimit);
	return rateLimit;
}

Cheers and thanks for the reply.

Derek

4

Forgot to comment on the utilization of the CAN bus. There shouldn’t be much on it, but there are some error from missing devices. I don’t know the utilization, but I will check that next.

Thanks again.

5

Following up now that most gremlins have been fixed.

CAN bus utilization was in the 30 - 40%, But we did find some hardware problems related to CAN. We had one Spark Max that seemed to be resetting the bus. The sleeps helped work around that, but once we replaced that Spark Max, those problems went away.
Steps to try:

  • Make sure all Spark Max controllers are at the same firmware version. 1.5.2 at the time of this writing. We had some at 1.5.1 and 1.5.2.

  • Do a factory reset on all the controllers using the Spark Max Client. We do this in the Java code too, but start fresh after the firmware updates didn’t hurt.

  • Check the bus with the Spark Max Client, this is where we saw resetting and repeating device addition messages.

  • Look for other problems, learn the blink color codes. One of the controllers had no feedback from the Neo. This was fixed by replacing the Spark Max. In this case, a yellow / purple blink was seen on the bad Spark Max.

  • Do a factory reset in your java code.

  • Burn the final controller settings in flash when you’re done setting them.

Cheers,
Derek

6

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.