f32::ZERO and f64::ZERO are not the additive identity

[Muon]

f32::ZERO and f64::ZERO are currently +0.0, which is not the additive identity. For non-NaN f32 and f64 under default rounding, the additive identity is -0.0 and not +0.0, since (-0.0) + (+0.0) == (+0.0) but (-0.0) + (-0.0) == (-0.0). (EDIT: interpreting == bitwise.)

[cuviper]

For some context, this was a pretty recent change in Rust for Sum:

• Change neutral element of <fNN as iter::Sum> to neg_zero rust-lang/rust#129321
• https://users.rust-lang.org/t/why-is-the-additive-identity-of-floats-0-0/128016

I still have some mild misgivings about it:

• Mathematically, zero has no sign, so arguments about additive identity are already a bit disconnected between pure mathematics and computer science.
• IEEE 754 defines -0.0 == +0.0, so even the equations you wrote will work either way. They're not bitwise-equal, but it's not clear to me that losing the negative sign is really any problem for additive identity. Why must we preserve it?
• +0.0 has the nice property that it is all 0 bits, so an initialization like vec![0.0; len] can use a faster calloc. But if you're doing that with ZERO and we change that to -0.0, it will have to write 0b1000... values throughout.

[Muon]

I agree about the performance implications, but the question really amounts to "should f32::ZERO and f64::ZERO be identity elements for the + operator?"

Regarding the mathematical misgivings, it is best to interpret -0.0 and +0.0 as infinitesimal elements and not zero.

[james-rms]

Considering that ZERO, the zero trait, zero() all are documented as returning "the additive identity", I think it makes sense to return -0.0 for the reasons explained above.

IEEE 754 defines -0.0 == +0.0, so even the equations you wrote will work either way. They're not bitwise-equal, but it's not clear to me that losing the negative sign is really any problem for additive identity. Why must we preserve it?

I don't really see the difference between this argument and "why should anyone care about the difference between +0.0 and -0.0" - If they don't care, then it should be fine to define ZERO as -0.0. If they do care, then they may be surprised when they add -0.0 to 0.0 and get 0.0.

+0.0 has the nice property that it is all 0 bits, so an initialization like vec![0.0; len] can use a faster calloc. But if you're doing that with ZERO and we change that to -0.0, it will have to write 0b1000... values throughout.

I think it's reasonable to say that those that want that property can initialize with vec![0.0, len()] directly.

[cuviper]

I think it's reasonable to say that those that want that property can initialize with vec![0.0, len()] directly.

They can't do that for a generic T: Zero though, e.g. for something like ndarray's zeros.

[cuviper]

Actually, let's ask -- @bluss @jturner314, how would you feel about this in ndarray's case? (or in general)

[cuviper]

nalgebra is similarly generic, @sebcrozet @milibopp what do you think?

[james-rms]

Another solution is to take the outcome of this discussion and use it to update the docstring, ie:

Defines an additive identity element for Self, 0. For floating-point numbers, this defines the additive identity for all values except for -0.0.
NOTE: zero() returns 0.0 instead of -0.0 for floating-point numbers so that it can be compatible with intializing large arrays using calloc, etc.

[bluss]

I don't think negative zeros would be appropriate for ndarray's use case, it would surprise all users and cost performance as noted. cc @akern40

(Unfortunately I think we'd need another Zero trait that gives positive zeros - in general, not thinking of ndarray, if this change was implemented so it would be "a bit" churny.)

[sebcrozet]

Same, I don’t think the enforcement of strict additive identity at the bit levels is worth the performance hit regarding allocating zero tensors. Personally I never think of zero as -0.0 and I can’t think of a edge-cases where we would benefit from Zero::zero actually returning -0.0. If the Zero trait is modified that way nalgebra would also have to switch to the different trait.
Adding the NOTE to the docs sounds like a practical solution.