You’re LabVIEW code looks appropriate; it writes encoder position to a network table entry every time a driver station packet is received and resets encoder position while a joystick button is pressed.
I agree with the other comments concerning electrical wiring and/or encoder function. A quadrature encoder relies on two signals to determine which direction to count. When only one signal is present the position count will alternate between two values.
An easy method to diagnose this electrical issue is to unplug one of the encoder signal wires (Chanel A/B) at a time and check to see if the behavior has changed. If unplugging one channel causes the position to stop changing while unplugging the other channel doesn’t change anything then you’ve isolated the issue. If that’s the case then it could be the encoder, the wiring, the roboRIO, or the digital pin assignments (selecting the correct digital I/O). Usually it’s the pin assignments or the wiring, but last night I experienced this symptom and traced it down to a bad encoder module.