Decimal precision in consts and use of double vs float

Technical Programming
1

One of my hot summer day activities in Hellaware, sorry Delaware is to read robot code. I’ve come across a few things that I’ve questioned. So I’m asking the developers to answer two simple questions

Why all of the highly precise constant defines?

Why all the doubles?

Constants

We all know that it’s faster to use things like the square root of 2 as constants rather than calling sqrt(2) lots of time.

Which leads to seeing things like this in the code


#define SQRT_2 1.414213562373095048801688724209698078569671875376948073176679737990733 
#define PI 3.14159265358979323846264338327950288419716939937510582097494459
//Efficiency or something

Lets take the Pi example. We all know that the diameter of a wheel times Pi gives you the circumference. An 8” wheel has the following circumferences depending on what value of Pi you use.

24” where Pi =3
24.8” where Pi = 3.1
25.12” where Pi is 3.14
25.13” where Pi is 3.1416

Assume we want to run the length of the FRC field (55’) and assume a 4’ long robot from the wall, the front wheel needs to move 51’.

24” where Pi =3 needs to do 25.5 rotations
24.8” where Pi = 3.1 needs to do 24.7 rotations
25.12” where Pi is 3.14 needs to do 24.4 rotations (*)
25.13” where Pi is 3.1416 needs to do 24.25 rotations

Using Pi=3 vs Pi as 3.1416 is a difference of 1.25 rotations, over 2’, but the difference in Pi at 2 decimal places 3.14 vs 3.1416 is 0.15 a rotation or about 3 ½ inches of actual travel after going 55 feet.

Six decimal places (3.141593 reduces the error to about 1/2”. Given friction, battery, carpets, windspeed, etc that is pretty amazing.

Key point is that is at 6 decimal places. Using more that that isn’t really helping that much, you are slowly closing in to the final point, but you are pushing the ability to make it any better.

So my first question is why the high decimal digit constants? This isn’t a space shot (and BTW they make tiny course corrections as they go). Are you really doing something with those extra digits? Or are you just fooling yourself that 3.141592653589793238462643383279 is way better than 3.141593?

Which brings on my second question, why all the doubles? A float on an ARM processor has about 7 decimal digits of accuracy. Unless you are doing a huge number (ie 10K+) of calculations, the rounding error isn’t going to be a problem. On the other hand if you are trying to do real time calculations, the cost of doubles in every calculation can be a burden.

While the sages on Stack Overflow say “Meh, just use double”, my second question is “Why all the calculations as double?”.

Thanks for your time on this hot summer day.

(*) On a 12’ VRC or 8’ VIQ field and 4” stock wheels, Pi = 3.14 is more than enough for the distances they are moving.)

2

With you all the way. When needed, I usually define pi as 4 * atan(1), but 22/7 is good to almost 1 part in 2500, and 355 / 113 is good to better than one part in 11,000. I once wrote an integer math routine to calculate atan2() in integer degrees based on integer inputs up to 1024, and it was really rather simple to get it right. Unless you’re calculating orbits until the next millennium or measuring the mass of a neutrino, there’s no need for THAT many digits.

Edit: In response to Brennon,
Start giving away points in anything and you’ll find out some day that you’ve given away too many. My integer atan2() was for an arduino, not a RIO, and I certainly don’t mean to advocate for writing such a routine for the RIO! That said, I can’t think of any reason to do significant amounts of double precision arithmetic on board an FRC robot - even (or especially) a 900 robot.

While on the subject, I’d like to call out Java* programmers who create a new object when the proper solution is to modify an existing one. Just because you **can **do something doesn’t mean you should.

  • there are probably also some C++ programmers who do this, but I’ve mostly seen it in Java.
3

My response would be “why not use a double?” In the context of FRC, I haven’t yet been in a bind where I couldn’t afford the memory. Further, Java defaults to using doubles for generic decimals anyway - using floats requires lots of casting. It keeps the code cleaner.

I noticed that you’re part of VRC - I can’t speak to that. It may be that a situation with much more restrictive memory constraints and not using Java may result in floats being more useful. If you’re going to go down this rabbit-hole further, I’d also suggest investigating fixed-point representations; they aren’t common except in embedded applications.

4

I use the digits of pi past the fourth as a kind of random number generator when I need to come up with a random number in my head. Aside from something like that though, no, later digits don’t add much value.

5

I’ve seen the ue of doubles in lots of C++ code, and it’s been mostly C++ code I’ve been browsing recently.

Java has some other issues with math, a little Google will get you case of where the JVM doesn’t make good choices. VRC control systems are pretty grunty for the size of the robot, the CPU in the VIQ system is pretty spiffy. But we put motor speeds (-127 -> 127) into ints rather than doubles.

I’m also a programmer of the GeeTwo base, I’ve done robotic stuff on Z80’s (lets build a sin table and do lookups rather than calculate). So I’m always looking at wasted cycles. In general I see programmers just doing things without looking at the overall effect. Which is why I need a 16GB of memory and 1Gb internet links to do simple things. As an example, CD, small footprint, loads in a heartbeat, bandwidth doesn’t eat my data plan for lunch. I have an ancient Wordpress site, it’s the same way. The new Wordpresss wallows like a hog waiting to be made into IRI cutlets.

I liked GeeTwo’s comment, don’t give points away. I’d rather spend the cycle on vision or making sure I have PID running where possible.

6

Here’s another real-world lesson in precision, from my career, and currently an issue. My work group is responsible for running performance predictions of US NAVY sensors over broad areas, interpreting these, and sending them forward to planners and operators. The predictions are run at double precision, but no one even pretends to trust these predictions to a resolution any finer than 0.1 dB, which is to say, about 2.3%, or about 5.4 binary digits of mantissa. About 15 years ago, “Jim” wrote some code to convert double precision binary quadrature code into single precision amplitudes. He intended it to last a year or so. We’re running that code to this day, because it reduces the data we need to send over the network by a factor of at least four, and the reduced data is worth as much as the original.
To intentionally misquote St Bernard of Clairvaux:

The road to hell is paved with unnecessary precision.

7

I agree that defining Pi past the 5/6th decimal place is totally unnecessary. You run into sensor limits far before software precision limits.

However, the tiny pure math part of me says “get everything as exact as possible”. That’s why I tried to work out the exact Hessian for the distance from a spline for about fifteen minutes, before I gave up and wrote a function that approximates it. I indulge this tiny math demon on my shoulder by using doubles and writing math.pi instead of 22/7.

I think using floats instead of doubles on the RIO is premature optimization. That’s a drop in the bucket compared to everything else going on. If it takes less than 20 ms to complete a robot loop, you’re letting that time go to waste anyway. If you’re taking more than 20 ms to complete a loop, then you need to first stop using Python, second, start thinking about optimizing hardware calls and etc, about thirty more items, then convert all your doubles to floats.

I can see the reason on Arduino and other embedded processors. I’ve run out of RAM on an Arduino before.

8

Regarding the extra precision in the original post, floats have about 7 digits of precision and doubles have about 15 digits of precision. If you’re using constexpr floats or doubles for arithmetic anyway, you might as well specify as many digits as can be represented in the type. Using a worse approximation like 22/7 in that representation is leaving free precision on the table.

Using floats instead is a premature optimization in one other way. When we transitioned WPILib’s C++ impl from using floats to doubles a year or two ago, we measured the time taken on a small floating-point benchmark on a roboRIO. When compiling with -Og (basically -O1 with some optimizations disabled for debuggability), the difference between floats and doubles was negligible.

We’ve tested other things as well in C++ like whether std::clamp() is better than writing out the if branches manually for clamping the integral term in PIDController.cpp. godbolt.org showed little difference in ARM assembly output for GCC 5.4 (the std::clamp() version was one fewer instruction iirc, but that’s probably just noise). Compilers are pretty smart nowadays, so write for readability and maintainability first, then optimize where you measure performance problems.

9

One place it really pays off to use floats rather than doubles is when your data is being transmitted wirelessly and you are really crunched for bandwidth where it doesn’t really matter if your values are off in the 10th decimal. This can happen quite readily in aerospace applications (among others). That said, I don’t think many teams are coming close to hitting their bandwidth limit unless they’re not being at all careful with video streams, so justifying floats over doubles in FRC seems rather difficult.

10

Another reason we chose doubles for wpilib, especially in the Java API is that constants in Java default to doubles, and don’t inplicitly get converted to floats. Also all the operations are only defined on doubles. So the amount of required casting was kind of insane. And as posted above, the speed performance was negliable.

11

Especially in a language like Kotlin (which we use), everything needs to be casted. Even passing 0 instead of 0.0 to an argument that requires a double would cause a compile-time error. Therefore, it’s a matter of convenience for us. And as iterated above, the performance difference is negligible.

12

Obligatory https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

TL;DR, it’s complicated. Unintuitively, there are real cases where doubles are faster than floats, and cases where floats have less error than doubles. Moreover, seemingly academic differences in precision can compound depending on what sequence of operations you enact on your values.

If it matters, you need to do some careful consideration of how your compiler/processor implements floating point operations. If it doesn’t matter, it doesn’t matter.

13

OP, awesome and thought-provoking question!

In my mind this rings true for the following reason:

–When you have no processing time or memory constraints, you will throw caution to the wind and not bother with determining how much precision is actually needed.
–When you hit processing and memory constraints, they are often sudden and… unexpected… leading to frantic slashing to see where you can make gains. Down-sizing variables while still guaranteeing you haven’t hosed functionality is an extremely frustrating endeavor.

Unless you have existing, good, 100% range coverage unit tests and can run stuff back to back, this will be a mind numbing endeavor.

If I were to see constants like that defined in code during a review, I would question why the “roll your own” approach was taken, rather than using some sort of built-in math library for the processor/language combo in question.

14

In many cases it is simpler and much more readable to create a new object instead of trying to keep track of what data in an object needs overwritten and deleted so it can be reused.

If you don’t NEED to do it, DON’T. Simple code is less buggy, easier to read, and easier to maintain. Just because you **CAN **reuse an object doesn’t mean you SHOULD.

15

FYI, on the theme of premature optimization, modern C++ compilers will do this optimization this for you.

They won’t know pi until you #include <cmath>, and at that point it is named M_PI. No real need to define either in robot code. For C++ anyway, not sure about java. Edit - for some dialects of C++, that is. Pretty sure GCC provides them in most cases, but your mileage may vary.

The number of digits might be for historical reasons - some systems had 80 bit or longer floating point registers (e.g. x87 - Wikipedia).

But yeah, unless you know what you’re doing float vs. double won’t make a difference. But if you do, there is potential, e.g. Auto vectorizing of ARM NEON float operation in gcc arm-linux-gnueabihf-gcc -mfpu=neon -O3 -S float_average.c arm-linux-gnueabihf-gcc -mfpu=neon -Ofast -S float_average.c · GitHub. This would make a lot more sense for, say, image processing instead of control code, but maybe there’s a chance it might do something if you’re lucky on other code :slight_smile:

16

ARM uses NEON for floating point numbers

It can execute up to 16 instructions for floating point at the same time (actual robot use cases would be 4 or 2 based on float or double. The 16 instructions is for specialized instructions)

Basically, it’s so fast that you really don’t have to worry about it. You would make much better use of your time optimizing somewhere else.

Not sure if this would make any difference on FIRST Robots, maybe vision, but here is an open multithreading library that can help execute code faster:

17

Not all ARM chips use NEON, and it’s not clear to me that the existing RoboRIO’s have the NEON instructions.

And I’m a fan of optimizing as we go. It’s like “swiss cheesing” robots. I’d rather design to not to, but if I have to, I’d like to cheese as I go, I hate to cheese on the eve of stop build day.

Are you advocating Pi for hundreds of places or doubles vs floats?

18

The RoboRIO does have NEON, and the compiler, at least for C++ definitely has it enabled. It does use the Soft Float ABI however, but the actual floating point calculations are definitely performed in hardware.

19

I’m just saying don’t worry about either. PI to 50 decimal places isn’t going to have an effect on the speed any more than 4 decimal places

The reason it’s there is because it was copied and pasted from another library (or is part of that other library) and they include it ‘just because’

If you were doing sine/cosine calculations that depended on each other, the error would add up. In a 2 minute match? Probably not enough, but in a 2 month application, you probably would start to see something

You are right that floats most of the time execute faster (depends on application, of course)

http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/

but remember you’re splitting hairs at that point (for FIRST Robots, anyways)

I’m not saying abuse doubles, but don’t be afraid of them. Trying to destroy every instance of a double will probably lead to more harm than good.

Like I said, there more places than just datatypes where you can explore efficiency

20

I was a bit put off by this at first, too. The thing is, the roboRIO really is much more like a slightly dated desktop than an embedded system in the traditional sense (well, excluding the FPGA, but we don’t get to play with that…). It has 256 MB of memory; it’s difficult to burn through that unless you’re doing something like vision processing. While it kind of offends my perfectionism, it really is usually more efficient to throw doubles at everything; you’re far more likely to be limited by build season time than computational resources.

Although, I have to say, it does slightly make me wish for the days of the PIC18-based IFI controllers, so that I could teach clever microcontroller tricks. (Bitfields! Inline assembly! Stack manipulation!)