We have wrestled with this same problem a bit (for a slightly different reason).
Over the past seasons, we have been increasing the steering speed in terms of module RPM. Basically, we are trying to reduce backlash more than trying to increase response speed but increasing the module speed by reducing gear ratio is good for both.
Our limit on the gear ratio (and therefore the number of stages of reduction) has always been controllability. In 2018 and 2019 we were running 150 RPM module speed with an RS550 (banebots) motor. We probably could have run faster, but we spent our time iterating other aspects of the designs instead.
Starting in 2020 we switched to the NEO550 we allowed us to run the steering control in the motor controller using the motor encoder itself for control rather than the absolute encoder. This allowed a much faster control loop without any jitter associated with the deadband due to geartrain backlash. We use the absolute encoder for the initial position, but then use the motor encoder for all control after that.
This allowed us to remove a stage from our steering gearset (we used a VP Lite for the steering reduction ahead of our final drive ratio). In 2020 and 2022 we had a total reduction from the NEO550 of 33.3:1 resulting in a theoretical free speed of the module of 330 RPM.
The other side of this coin is that we have reduced the steering torque available. This is where the math comes in.
If you look at our module, you can see that there is a VP gearbox between the drive motor and the bevel gear. When this motor applies a torque to drive the wheel, this torque will also try to spin the module due to the reaction torque at the 90 degree bevel gears. If the wheels were unable to turn (such as when you are in a pushing match) and the motor were at max stall torque, the reaction trying to spin the module would be the motor’s stall torque times the gear ratio of that gearing between the motor and the bevel. You should be able to calculate the motor reaction torque for your system pretty easily as it is is always the motor torque times the gear ratio of the gears before the bevel gear.
With our gearing, we run either 3:1 or 4:1 before the bevel and 2:1 after the bevel gear (in 2018 and 2019 we were running 3:1, but at the suggestion of another team we tried 4:1 this year to get better acceleration).
So, if you take the stall torque of our NEO motor @40A, times this gearing (we will take the worst case which is the 4:1), you get 2.92 ft-lb of torque trying to spin the module.
I believe that this is the highest load that the steering motor needs to react. This is also the reaction torque that you are showing in your diagram.
Taking the NEO 550 steering motor, you have 0.215 ft-lbs of stall torque available @30A. Apply the 33.3:1 ratio (10:1 in the VP gearbox and 100:30 in the final drive ratio) and you get 7.15 ft-lbs of torque available to resist the drive motor. This is 2.45 times the maximum torque from the drive motor described above. So, this gives us a decent factor of safety and ensures that we are not running the motor at excessive amperage at stall as we are resisting the torque of the drive motor. Taking the 30 Amps divided by 2.45, we get 12.25 Amps. REVs testing of the NEO550 says that it can sustain up to 20 amps continuously (over a minute) without failing, so 12 amps seems pretty safe.
So, my first question about your problem is, have you done the similar math to determine what sort of torque factor you have (stall torque divided by working torque to react the drive motor)?
If you have a decent margin there, then my second question would be have you measured how much backlash you have in your greartrain between the steering motor and the module azimuth. It could very well be that it is the backlash in the geartrain rather than the backdriving of the motor that are causing your issues.
Depending on which of these two issues is causing your problem, you should be able to fix it with some clever programming.
If you are backdriving your motor, then you can just drive the motor against the backdriving force. Since you have now done the math and you know how much torque will be applied to the steering motor, you can apply sufficient amperage to generate torque in the opposite direction to oppose the tendency for the module to rotate.
If you have too much backlash, then you would want to rotate the motor in one direction or the other to “consume” the deadband.
You can program these steering motor behaviors to occur at the same time that you are applying the drive torque before the steering inputs are calculated from the encoder.