I’m aware of some issue with floating point precision, but I don’t understand this one:
When storing 9138497.986 as a Float(), it appears to be getting truncated to “9138497”. Can someone explain why?
Forcing it to print 3 decimal places gives “9138497.000”
Examples:
Float.greatestFiniteMagnitude
// 3.402823e+38
let f1 = Float(9138497.986)
String(format: "%.3f", f1)
// "9138498.000 <-- Why is this happening?
// divide by 10 to show decimals
let f2 = Float(9138497.986 / 10.0)
String(format: "%.3f", f2)
// 913849.812
// `Double` is fine
let d1 = Double(9138497.986)
String(format: "%.3f", d1)
// "9138497.986
// I found this when using `SCNVector3()`, which uses `Float` internally
import SceneKit
let v = SCNVector3(216877.354, 9138497.986, 2878.604)
v
// x: 216877.4, y: 9138498, z: 2878.604
5
A Float is a 32 bit floating point number. It has an exponent that is 8 bits in size and a significand that is 23 bits in size. This gives you 24 binary digits of precision (including the implied leading 1). That is equivalent to a little over seven decimal digits and your number has seven decimal digits before the floating point. In simple terms, the full precision of your number won’t fit inside a Float
Here’s a bit of code that confirms this:
let f1 = Float(9138497.986)
print(f1.nextUp) // The next bigger number representable as a Float
print(f1.nextDown) // The next smaller number representable as a Float
print(f1.ulp) // The gap between Floats at this magnitude
Prints out
9138499.0
9138497.0
1.0
Note that these three numbers are, in reality not likely to be exact whole numbers but there is an interaction with the formatting code such that the rounding makes the whole number the closest expressible representation to what they really are.
The absolute precision of floating point numbers varies — that’s essentially what “floating point” means.
For single precision, for values between 1.0 and 2.0, the precision is about 1.2×10-7. Between 2.0 and 4.0, it’s half that. Between 4.0 and 8.0, it’s half that again. By the time you get to the range 4194304–8388608, the precision is just 0.5 — that is, you can represent the numbers 5000000 and 5000000.5, but you can not represent, say, 5000000.3 — it gets rounded off to 5000000.5.
And then when you get to the range 8388608–16777216, the precision is 1 — you can’t represent numbers with fractional parts at all; everything gets rounded off to the nearest integral value.
Going further, in the range 16777216 to 33554432, the precision is 2: you can’t represent odd numbers. Between 33554432 and 67108864, you can only represent numbers that are an exact multiple of 4. And so on.
Your number, 9138497.986, is in the range where the precision is 1. Therefore, it typically gets rounded off to 9138498. You may have heard that single-precision floating point has the equivalent of “about seven decimal digits” of precision, and this is a case where that adage happens to work perfectly: the seven digits 9138498 are all you get; there are no digits left over for a fractional part.
(At one point in your answer you said you saw 9138497, which would be correct for truncation, but wrong for round-to-nearest.)
All the values I’ve given so far are for single precision, or “float”. For double precision, there is literally about twice as much precision, which means there’s no problem representing a number pretty close to 9138497.986, but if you go to still bigger numbers, you’ll see the same pattern.
In double precision, between 1.0 and 2.0 the precision is about 2.2×10-16. You have to go up to the range 4503599627370496–9007199254740992 before the precision drops to 1, but above that, there are integral values you can’t represent in double precision, either.