Look:
A = 11.531681447264651 B = 10.276344990825836 C = 0.11732116415316027 D = 10.7
Now, try it:
A - B -C*D
Result: 6.661338147750939e-16
- Why a result so small?
But, lets round it:
round(A,1) - round(B,1) -round(C,1)*round(D,1)
Result: 0.12999999999999923
Great result! If a try in calculator
11.5 − 10.3 −0,1×10.7 = 0.13
2. Why 6.6e-16
before and now 0.129999
?
Other thing weird:
round(A,2) - round(B,2) -round(C,2)*round(D,2)
Result: -0.03399999999999981
-
Why negative now?
-
My experimental results is with precision 1, so, should I put all A, B and C numbers to precision 1 before do this math, right?
I tried all variations, and solved using round(x,my_precision)
before the math, but I would like to understand what is happening
jarlisson is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.