View Single Post
  #8   Spotlight this post!  
Unread 07-23-2015, 10:38 PM
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,748
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Summer Labview Challenge 2

The strange bug happens because by the time you have made it to the penny calculation, the remainder is no longer a full two cents. If you display it with about 15 digits of precision, it is truly just a bit below, so the node returns almost a full penny as a remainder. This is largely due to the fact that floats cannot store base ten fractions precisely. The mixup rounds the value so that the small cumulative error resets. A different fixup would be to convert the initial value to units of pennies instead of dollars.

Greg McKaskle
Reply With Quote