The servo signal input to the Jaguar controls the voltage output. There isn't built-in closed-loop control when using the servo signal input.
You can definitely connect an optical encoder to the Jaguar. Please see the Getting Started Guide at
www.luminarymicro.com/jaguar for an overview on all the possible control modes that the Jaguar supports.
You can connect encoders, limit switches, and/or analog potentiometers (or other analog signals) to the Jaguar inputs. Using CAN, you can change the mode to control voltage, speed, position (like a hobby servo, for example, imagine a CIM motor that holds position like a servo!), and current and then change parameters for P, I, and D so that the Jaguar does all the closed-loop control for you.
-David