Defining zero() seriously

[jiahao]

(This issue is a follow-up to #34000 #28854 #31303)

The docstring for zero(x) says

Get the additive identity element for the type of x

However, defining this additive identity precisely seems to be confusing, especially for union types, and in the presence of type promotion rules. The rest of this issue is dedicated to exploring more precisely what the semantics of zero(T) is (to help figure out what exactly the semantics should be).

1. In many situations, zero(::Union) is undefined when it could be defined. Intuitively, one might expect that the presence of multiple zeros might complicate the definition. However, type promotion seems to allow for a uniquely defined result. If we take the definition above seriously, the statements

julia> Float16(0.0) + Float32(1.0) === Float32(1.0) #works for Float32
true

julia> Float16(0.0) + Float16(1.0) === Float16(1.0) #and for Float16
true

would argue that zero(Union{Float16,Float32}) === Float16(0.0) (it is currently undefined).

2. A counterargument to the above would be that Int(0) is also a valid choice for zero(Union{Float16,Float32}), since

julia> 0 + Float16(1.0) === Float16(1.0) # works because of type promotion
true

However, the statement above would also imply that Int(0) is a valid choice for zero(Float16).
We can exclude this value by requiring zero() to be endomorphic, i.e. zero(::T) :: T, and by extension zero(T) :: T.

3. Int(0) appears to be the result of zero(T) wheneverT <: Union{Missing,Number,Complex}, even though it is not always the unique (or even correct) additive identity in T:

julia> zero(Union{Int,Complex})
0

julia> zero(Union{Real,Complex}) #due to type promotion, see #2
0

julia> 0 + Complex{Bool}(0) === Complex{Bool}(0) # Counterexample: actually Complex{Int}(0), so Int(0) is not an additive identity over Complex
false

julia> Int(0) + Int16(0) === Int(0) #Counterexample: Int16 <: Real but adding Int(0) to it promotes to Int
true

julia> Int(0) + false === Int(0) #Counterexample: Bool <: Real but adding Int(0) to it promotes to Int
true

---

Proposal

1. zero(T) for non-union types should be defined as the unique additive identity such that
   a. zero(T)::T, i.e. zero() is an endomorphism, and
   b. The identity zero(T) + zero(T) === zero(T) is satisfied for all T.

2. zero(Union{T,S}) === zero(T) when zero(T) + zero(S) === zero(S), taking into account type promotion rules.

3. If no unique endomorphic zero exists, then zero(T) should be undefined, except for zero(Bool) === false as a special case.

This definition would preserve the endomorphic property for leaf types, like zero(Float16) === Float16(0.0).

This definition would preserve the result for zero(Bool) === false, to be consistent with current arithmetic semantics of Bool.

This definition would preserve the semantics of #28854 for zero(Missing) === missing.

This definition would define zero(Union{Float16,Float32,Float64}) to be Float16(0.0) (currently undefined).

This definition would define zero(Union{Float16,Int16}) to be Int16(0) (currently undefined).

This definition would change zero(Real) to be false (and similarly in Case 3 above; currently Int(0)); arguably a bug fix.

This definition would change zero(Union{Bool,Int}) to be false (currently Int(0)); arguably a bug fix.

[jiahao]

The two cases that would change under the proposal above would be preserved if we defined zero(Bool) === Int(0), which is arguably the actual arithmetic semantics being described. Might be less of a breaking change to pick Int(0) over false.

[oscardssmith]

Would it make sense to define a zero singleton that just adds to everything the way we want it to, or would that break type stability?

[piever]

I hope this does not add too much noise to the discussion, but a similar need (of a "symbolic zero" object z, such that a+z === a for all a), was expressed in FluxML/Zygote.jl#275. I imagine this would be more useful now that in the past, as the type instability is not so bad for performance due to small union optimizations.

[StefanKarpinski]

Would it make sense to define a zero singleton that just adds to everything the way we want it to

That's already essentially what false is.

[MasonProtter]

As it happens I was just thinking about this here: FluxML/Zygote.jl#329 (comment)

I think it would make sense to have an analogue to LinearAlgebra.I, perhaps LinearAlgebra.Z (which I guess we can't export until 2.0) which has the right properties of zero just like UniformScaling. Call it UniformZero or whatever.

@StefanKarpinski

That's already essentially what false is.

Maybe for Numbers, but not anything else:

julia> false + [1 0; 0 1]
ERROR: MethodError: no method matching +(::Bool, ::Array{Int64,2})

Also, as @MikeInnes has pointed out, you can't dispatch on false which is sometimes desirable.

[jiahao]

Another use case I ran into is defining zero(::Nothing) = nothing - permissible if considering that Nothing has only one element (nothing), then one can define the trivial algebra over it. It turned out to be useful in an edge case with the method

ForwardDiff.partials(x, i...) = zero(x)

[nalimilan]

I'd rather not define any arithmetic operation on nothing. Its main raison d'être is that it throws an error as soon as you try to do something with it, which helps catching bugs. We have missing when you need propagation.

[kmsquire]

Would it make sense to define a zero singleton that just adds to everything the way we want it to

That's already essentially what false is.

So, this suggests that, when writing code that is supposed to be generic over numbers, we should always use false instead of 0? 🧐

[StefanKarpinski]

Yes, that's essentially what it suggests. And true for the generic unit value.

[kmsquire]

But doesn’t using false to mean “the additive identity” (0) and true as “the additive unit value” (1) seem like we’re overloading the meaning of what are traditionally just Boolean values?

[MikeInnes]

I think it seems more sensible if you think of Bool as equivalent to UInt1; boolean operations are just special cases of the bitwise operations.

[kmsquire]

Even if it can be justified, it still seems strange to me to write

a += true

in order to add 1 to a in a generic way.

[StefanKarpinski]

I don't disagree; what would you prefer to write?