Go to Post To all programmers: you are Geniuses! - Ashley Weed [more]
Home
Go Back   Chief Delphi > Technical > Programming > NI LabVIEW
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
 
 
Thread Tools Rate Thread Display Modes
Prev Previous Post   Next Post Next
  #9   Spotlight this post!  
Unread 07-24-2015, 10:42 PM
Jonathan L. Jonathan L. is offline
Registered User
FRC #1094
 
Join Date: Jan 2013
Location: St. Louis MO
Posts: 60
Jonathan L. is a jewel in the roughJonathan L. is a jewel in the roughJonathan L. is a jewel in the roughJonathan L. is a jewel in the rough
Re: Summer Labview Challenge 2

Here's Part 2.

Quote:
Originally Posted by Greg McKaskle View Post
The strange bug happens because by the time you have made it to the penny calculation, the remainder is no longer a full two cents. If you display it with about 15 digits of precision, it is truly just a bit below, so the node returns almost a full penny as a remainder. This is largely due to the fact that floats cannot store base ten fractions precisely. The mixup rounds the value so that the small cumulative error resets. A different fixup would be to convert the initial value to units of pennies instead of dollars.

Greg McKaskle
Thank you for explaining. So is this something that can be fixed by NI or just a quark in programming? I remember there was something similar last year with the equal function but that seems to have been fixed.
Attached Thumbnails
Click image for larger version

Name:	Summer LabVIEW Challenge 2 Part 2.png
Views:	10
Size:	56.9 KB
ID:	19228  
Reply With Quote
 


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 06:07 PM.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi