It seems like your program isn’t updating the motor values when it’s at zero. If you are able to calibrate but than it goes back to flashing orange that probably means motorsafety is killing the outputs for some reason.
If you only update the motor values if the joystick value changes, that might result in this type of problem.
Are you establishing all of them as Victors and not as Jaguars in code?