|
Re: Summer Labview Challenge 2
Take a look at the Wikipedia article on floating point representation for a description of the issue we face:
https://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion _and_rounding
Quote:
For example, the decimal number 0.1 is not representable in binary floating-point of any finite precision; the exact binary representation would have a "1100" sequence continuing endlessly: e = −4; s = 1100110011001100110011001100110011...,
When rounded to 24 bits this becomes
e = −4; s = 110011001100110011001101,
which is actually 0.100000001490116119384765625 in decimal.
|
__________________
"Rationality is our distinguishing characteristic - it's what sets us apart from the beasts." - Aristotle
Last edited by Mark McLeod : 07-25-2015 at 10:13 AM.
|