The smallest number that can be stored by an IEEE-754 32-bit float is 2^-126 ~ 1.18e-38. Why then do I get the following output?
print(np.float32(1e-39))
> 1e-39
Through experimentation, it looks like the smallest float that is actually truncated to 0 is around 1e-46. It is able to perform computation on these small numbers too. For example:
print(np.float32(2*1e-45))
> 2.802596928649634e-45
Obviously there is a very large truncation error here, but it is nevertheless clearly performing computation on these very small numbers. What’s going on? Does the numpy float32 not conform to IEEE-754?
2