[Jared Russell]

Almost. It is MUCH easier to re-center the joystick around 0 first:

int p3_x_mod = (int)p3_x - 127;

You should then fix the "b" part of y=mx+b to 127 so that no matter what, not moving the joystick is neutral. Also, with integers, (11/51) will be evaluated first as ZERO, screwing everything up. Using p3_x_mod, which is an int, in the expression below also makes sure that you don't overflow the size of an unsigned char.

Try:

pwm16 = (11*p3_x_mod/51) + 127;

Your final code would look something like:

int p3_x_mod;
...

if( p3_sw_top )
{
  p3_x_mod = (int)p3_x - 127;
  pwm16 = (11*p3_x_mod/51) + 127;
}
else
{
  pwm16 = p3_x;
}

The code you wrote works fine on paper, but it suffers from both the rounding to zero and size overflow problems.