Velocity PID(F) Best Practices - To Integrate, or Not To Integrate?

[Oblarg]

There seems to be a general lack of agreement about the best way to handle velocity PID loops.

In particular, in comparison to position PID loops, velocity PID loops present us with a problem:

In a position PID loop, there is a natural integrator between the loop output (motor voltage, which corresponds roughly to velocity) and the setpoint, since the position is the integral of velocity.

On the other hand, in a velocity loop, there is no such integrator.

Thus, the two loops behave, at a fundamental level, extremely differently. This much is known to basically anyone who has ever used a velocity PID loop.

One solution to this, currently employed by WPILib, is to integrate the output of a velocity loop (save for the feedforward term, for obvious reasons). This serves to make the relationship between the loop output and the setpoint more closely resemble that of a position loop. Effectively, this is the same as turning the 'p' term into what was, pre-integration, the 'i' term, and turning the 'd' term into the 'p' term (since, per the fundamental theorem of calculus, the integral of the derivative of the error is simply the error itself).

However, I've also heard many teams state that, with use of a feedforward term, there is no need to do this and simply using a non-integrated loop predominantly tuned with the P term suffices.

I, myself, am not quite sure what to make of this, for several reasons:

For the first method, the 'd' term in an integrated velocity loop (i.e. the 'p' term in a non-integrated loop) really doesn't behave entirely like the 'd' term in a position loop; the aforementioned fundamental-theorem-of-calculus argument only indicates that their effect on the endpoint behaves similarly up to some constant offset - and this constant may be nonzero, depending on how 'd' responds to changes in setpoint (e.g. if my setpoint is 1, and my measured endpoint is 0 and has not yet started to change, the resulting 'd' term of an integrated velocity loop will be some positive constant, but the 'd' term of a position loop would be 0 - only the effects of the changes to the two terms from this point are analogous). Moreover, motor voltage does not precisely correspond to velocity (if it did, we'd have no need of a control loop!), and so this causes differences in behavior, as well.

However, for the second method, the 'p' term of a non-integrated velocity loop certainly does not behave like the 'p' term of a position loop, so tuning around 'p' in a non-integrated loop (even in the presence of a feedforward term) has never made much sense to me. I can totally believe that it works, but it does seem like it would be closer (but, as per the previous paragraph, not analogous) to tuning around the the 'd' term of a position loop rather than the 'p' term, and that is confusing.

I know many teams (including mine) tune around the 'p' term of an integrated loop (i.e. the 'i' term of a non-integrated loop), even when using feedforward. This seems to work very well for us, but I've seen many teams say that they do not use this term at all for their velocity loops.

So, is there any consensus about this, at all?

[AdamHeard]

A key point is that the 1khz loop available on the Talon's makes the 2nd method much more robust than it would be running at a much slower rate.

Industry does it quite a few ways, and you're correct there isn't a consensus.

[Jared Russell]

Originally Posted by Oblarg

However, for the second method, the 'p' term of a non-integrated velocity loop certainly does not behave like the 'p' term of a position loop, so tuning around 'p' in a non-integrated loop (even in the presence of a feedforward term) has never made much sense to me. I can totally believe that it works, but it does seem like it would be closer (but, as per the previous paragraph, not analogous) to tuning around the the 'd' term of a position loop rather than the 'p' term, and that is confusing.

Bang-bang control is a proven and effective method of velocity control (especially for systems with considerable inertia, like flywheels). The second method (which is more common in mobile robotics) is just refined bang-bang.

So why does it work well for velocity, but (practically) nobody uses bang-bang for precise position control?

Inertia is the dominant factor in the dynamics of most FRC-scale systems that you may want to control. Inertia has a first order effect on a change in velocity, but a second order effect on a change in position. Because the effect is squared in position control, you usually can't tune a proportional controller to track a reference position without steady state error and without excessive oscillation simultaneously - if you ratchet up the gain to eliminate steady-state error, you become underdamped and begin oscillating. This happens because at some point, when your output crosses your reference, your loop reverses signs and commands movement back in the other direction. But the system inertia keeps you going in the wrong direction for a while until you can get back to zero speed.

(So you then use the I and D terms to compensate for this aspect of your dynamics by slowly increasing the output to eliminate steady-state error, or prematurely dampening the output to compensate for inertia and minimize overshoot, respectively.)

In a velocity controller, inertia has a linear effect. If you change the sign of the control output, you expect your system to begin accelerating in the opposite direction immediately (modulo small but inevitable delays in your system). This means you can get away with really high gains without experiencing stability problems. Ex. a bang-bang controller essentially has an infinite proportional gain yet remains stable for many systems.

The only time I have seen the "integrated" PID controller formulation used in industry is on conveyors or machinery that are designed to basically move at a constant speed all the time. Speed of response is not important in this use-case, but outputting a very smooth velocity signal to help improve the service life of mechanical components is. In other words, these controllers respond to disturbances like slowly wearing bearings or oil quality changes rather than dynamic loading and unloading.

[Oblarg]

Originally Posted by Jared Russell

In a velocity controller, inertia has a linear effect. If you change the sign of the control output, you expect your system to begin accelerating in the opposite direction immediately (modulo small but inevitable delays in your system). This means you can get away with really high gains without experiencing stability problems. Ex. a bang-bang controller essentially has an infinite proportional gain yet remains stable for many systems.

So, basically, if you're using feedforward, you can make the 'p' gain in a non-integrated velocity controller big enough that the fact that it can't theoretically push the velocity all the way to the setpoint (even as t->infinity) is essentially moot, without running into stability issues?

[ozrien]

Originally Posted by Oblarg

So, basically, if you're using feedforward, you can make the 'p' gain in a non-integrated velocity controller big enough that the fact that it can't theoretically push the velocity all the way to the setpoint (even as t->infinity) is essentially moot, without running into stability issues?

I think that's fair. In FRC it is generally acceptable that the velocity servo is oscillating (underdamped) by a few RPM above and below your target-velocity, and in exchange you get a fast response when target velocity changes, with no risk of integral-windup.

Similar thread for those interested: https://www.chiefdelphi.com/forums/s...d.php?t=152106

Just curious, did your team attempt a velocity servo and wasn't successful?

[Oblarg]

Originally Posted by ozrien

Just curious, did your team attempt a velocity servo and wasn't successful?

Not yet. We made our first foray into velocity control last year, and wrote our own integrator to slap onto WPILib's PIDcontroller object (WPIlib had not yet implemented the integrator they have now) because that was the solution that made the most conceptual sense to us. This past summer, when WPIlib did implement that, we added a feedforward term, but still tuned largely around the 'p' term of the integrated loop ('i' term of a non-integrated loop).

This preseason we are experimenting with doing PID control on the talons, and so the main question is whether we should tune around 'p' or 'i'. We're also still wrestling with technical issues, so we haven't experimented with either on the Talon quite yet.

[Oblarg]

Just thought I'd report back - we just got a flywheel working with a PF loop, basically exactly as described above. Indeed, there is plenty of headroom to make the P term big enough that the steady-state error is negligible (plus or minus ~.2 RPS) without the system becoming unstable. We'll see about tuning our drive this way next, though we need a few more talons to do that.

Just a note about PID control on the talons: god, the handling of the units in the java WPILib code is terrible. "Native units" (which are noted to be either per 10ms or per 100ms depending on which source you check; they are actually the latter) are used everywhere *except* for the set and getsetpoint functions, which use RPM. The comments helpfully mention that the latter function returns a value in "appropriate units." Also, the output scaling for F gain is -1023 to 1023, which is a sort of odd choice.

On the plus side, the loop works great.

I hope, before too long, to try tuning a variety of systems using both strategies and actually compare the performance.

[Jared Russell]

Originally Posted by Oblarg

Just a note about PID control on the talons: god, the handling of the units in the java WPILib code is terrible. "Native units" (which are noted to be either per 10ms or per 100ms depending on which source you check; they are actually the latter) are used everywhere *except* for the set and getsetpoint functions, which use RPM. The comments helpfully mention that the latter function returns a value in "appropriate units." Also, the output scaling for F gain is -1023 to 1023, which is a sort of odd choice.

I agree wholeheartedly...I'm hopeful that this will be improved for 2017.

[Oblarg]

So, here's a graph of our flywheel velocity error (units in RPS) versus time during a shooting test. The negative spikes are slowdown while the flywheel is in contact with the ball. The velocity setpoint was 50 RPS.



As we can see, consistent with the above discussion, there is steady-state error, but it is very small.

We are wondering if teams bother to introduce any integral gain to eliminate the steady-state error, and if this performance is consistent with what other people have seen using similar methods.

[Bryce2471]

Originally Posted by Oblarg

So, here's a graph of our flywheel velocity error (units in RPS) versus time during a shooting test. The negative spikes are slowdown while the flywheel is in contact with the ball. The velocity setpoint was 50 RPS.



As we can see, consistent with the above discussion, there is steady-state error, but it is very small.

We are wondering if teams bother to introduce any integral gain to eliminate the steady-state error, and if this performance is consistent with what other people have seen using similar methods.

Great data. Thanks for that contribution. I have a couple questions:
How long did you wait between shots?
How much was the steady state error effected when you changed the set point.
How much time, or how many iterations did you spend tuning the Kf value? How about Kp?

Thanks again.

[Jared Russell]

Originally Posted by Oblarg

So, here's a graph of our flywheel velocity error (units in RPS) versus time during a shooting test. The negative spikes are slowdown while the flywheel is in contact with the ball. The velocity setpoint was 50 RPS.



As we can see, consistent with the above discussion, there is steady-state error, but it is very small.

We are wondering if teams bother to introduce any integral gain to eliminate the steady-state error, and if this performance is consistent with what other people have seen using similar methods.

We didn't use integral gain, but we did use some derivative gain to lessen the initial overshoot, and a 36V/s rate limit to smooth things out a bit:
https://github.com/Team254/FRC-2016-...ants.java#L152

(Talons were in coast mode as well)

We allowed firing whenever our flywheel was within 100rpm (1.67rps) of our setpoint because we found it made no practical difference to our shot trajectory. In practice, most shots we took were within 20rpm of our setpoint. If you are within .1rps (6rpm) steady-state, I'd say you're already in good shape.

[Oblarg]

Originally Posted by Bryce2471

Great data. Thanks for that contribution. I have a couple questions:
How long did you wait between shots?
How much was the steady state error effected when you changed the set point.
How much time, or how many iterations did you spend tuning the Kf value? How about Kp?

Thanks again.

1) We didn't time this precisely, but it was on the order of 10 seconds.
2) We haven't rigorously tested this yet, will report back once we have data
3) We tested our flywheel speed at max voltage, and then simply naively set Kf to (max output)/(max speed). We did not test across multiple battery voltages or setpoints - if anyone has suggestions for a good systematic way to do this, it'd be appreciated.
4) We started Kp pretty small and then doubled it until we started to see oscillations, at which point we set it to halfway between that and the previous value. There were a couple more iterations after that, I think, but I don't recall exactly how many.

All in all, it was a pretty "rough" tuning procedure - we were mostly interested in getting it working, with the understanding that we can fine-tune it later.

Originally Posted by Jared Russell

We didn't use integral gain, but we did use some derivative gain to lessen the initial overshoot, and a 36V/s rate limit to smooth things out a bit:
https://github.com/Team254/FRC-2016-...ants.java#L152

(Talons were in coast mode as well)

We allowed firing whenever our flywheel was within 100rpm (1.67rps) of our setpoint because we found it made no practical difference to our shot trajectory. In practice, most shots we took were within 20rpm of our setpoint. If you are within .1rps (6rpm) steady-state, I'd say you're already in good shape.

Thanks a bunch for all of your input - it's greatly appreciated. I'll add these to the list of things to try.

Hopefully we'll find some time to sit down and really play with different tuning schemes, and record the data.

[calcmogul]

I'm the person who implemented the velocity PID controller in WPILib last summer. If I understand correctly, an integrator is necessary and sufficient to eliminate steady-state error in controlling a DC motor with a velocity PID controller in all cases. Here's some math:

I'll be using the model for a DC motor from http://ctms.engin.umich.edu/CTMS/ind...stemModelin g, which has the following transfer function from voltage (V) to angular velocity (theta_dot):

P(s) = theta_dot(s)/V(s)=K/((Js+b)(Ls+R)+K^2)

First, we'll try controlling it with a P controller defined as:

C(s) = Kp

When these are in unity feedback, the transfer function from the input voltage to the error is:

E(s)/V(s) = 1/(1+C(s)P(s))
E(s) = 1/(1+C(s)P(s))*V(s)
= 1/(1+Kp*K/((Js+b)(Ls+R)+K^2))*V(s)

(See http://ctms.engin.umich.edu/CTMS/Con...motorBlock.png for a diagram. I'm assuming that theta_dot_ref maps to the input voltage with a constant gain and that the measured signal is used as-is in the error calculation, so H(s) = 1).

In this case, we'll apply a step input, so V(s) = 1/s.

E(s) = 1/(1+Kp*K/((Js+b)(Ls+R)+K^2))*1/s

To find the steady-state value of a transfer function, we can apply the formula:

e_ss = lim_s->0 s*E(s)

e_ss = lim_s->0 s*1/(1+Kp*K/((Js+b)(Ls+R)+K^2))*1/s
= lim_s->0 1/(1+Kp*K/((Js+b)(Ls+R)+K^2))
= 1/(1+Kp*K/((0+b)(0+R)+K^2))
= 1/(1+Kp*K/(bR+K^2))

Notice that the steady-state error is non-zero. To fix this, an integrator must be included in the controller:

C(s) = Kp+Ki/s

Same steady-state calculations as before with the new controller:

E(s) = 1/(1+C(s)P(s))*V(s)
= 1/(1+(Kp+Ki/s)*K/((Js+b)(Ls+R)+K^2))*1/s

e_ss = lim_s->0 s(1/(1+(Kp+Ki/s)*K/((Js+b)(Ls+R)+K^2))*1/s)
= lim_s->0 1/(1+(Kp+Ki/s)*K/((Js+b)(Ls+R)+K^2))

multiplying by s/s:

= lim_s->0 s/(s+(Kp*s+Ki)*K/((Js+b)(Ls+R)+K^2))
= 0/(0+(0+Ki)*K/((0+b)(0+R)+K^2))
= 0/(Ki*K/(bR+K^2))

the denominator is non-zero, so e_ss = 0.

So mathematically speaking, an integrator is required to eliminate steady-state error in all cases for this model. If you don't want to use one, a feedforward can eliminate most if not all of the error if chosen carefully. Given that, there is always uncertainty, hence why feedback control exists.

Oblarg is correct that the behavior of the velocity PID tuning constants doesn't match the displacement PID. I tried to get something with close enough dynamics that teams could use a similar tuning procedure for both (except in the velocity case, there's no steady-state error so I made Ki not do anything).

[AdamHeard]

Originally Posted by calcmogul

So mathematically speaking, an integrator is required to eliminate steady-state error in all cases for this model. If you don't want to use one, a feedforward can eliminate most if not all of the error if chosen carefully. Given that, there is always uncertainty, hence why feedback control exists.

In the practical sense most teams are just looking for consistent RPM within a tight band near the setpoint.

If the system inertia is large enough compared to the motor power and the control frequency high enough you'll get high frequency oscillations of low magnitude around the setpoint (as Jared and Oblarg detailed above).

So, you need an integrator to eliminate steady state error in a theoretical sense, but not necessarily in a practical sense depending on the system.

[Jared Russell]

Originally Posted by calcmogul

Oblarg is correct that the behavior of the velocity PID tuning constants doesn't match the displacement PID. I tried to get something with close enough dynamics that teams could use a similar tuning procedure for both (except in the velocity case, there's no steady-state error so I made Ki not do anything).

This is totally reasonable, and in practice you need an integrator if your loop rate is slow or you are concerned about the effects of rapid oscillation on the lifetime of your mechanical components.

Originally Posted by calcmogul

e_ss = 1/(1+Kp*K/(bR+K^2))

e_ss approaches zero as Kp approaches infinity, and the closed-loop transfer function is stable for all positive values of Kp.