[Jared]

I would set up something that limits how quickly your velocity setpoint can change when you are slowing down. You could put this in the part of your code where you're mapping joystick position to desired velocity.

For example, you could have something like:

//set currentSetpoint here
if(currentSetpoint - lastSetpoint > .1){
    currentSetpoint = lastSetpoint - .1;
}
//set your PID controller here
lastSetpoint = currentSetpoint;

You could replace .1 by how much you want the maximum change in velocity setpoint per 20ms to be. You'd also need to do something clever for when you're going in reverse and slowing down, but the idea is the same.