Thread: Floating point math zero
View Single Post
  #2   Spotlight this post!  
Unread 09-02-2007, 00:08
Jared Russell's Avatar
Jared Russell Jared Russell is offline
Taking a year (mostly) off
FRC #0254 (The Cheesy Poofs), FRC #0341 (Miss Daisy)
Team Role: Engineer
  
Join Date: Nov 2002
Rookie Year: 2001
Location: San Francisco, CA
Posts: 3,077
Jared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond repute
Re: Floating point math zero
At first I suspected that the problem had to do with integer overflow at this step:

(float)(prettyJoyX*prettyJoyX + prettyJoyY*prettyJoyY)

However, assuming prettyJoyX and prettyJoyY are bounded at +/- 127, this wouldn't cause an overflow. Obviously, it also would never be negative. If it isn't the case that those variables are bounded, then it could be possible that you are overflowing, causing the data to wrap around to a negative number and invalidating "sqrt".

I will continue investigating.

For what values of prettyJoyX and prettyJoyY do you get zero?