Symbol ⊗ [Unicode 'CIRCLED TIMES' (U+2297)], typed \otimes in Julia, as the default shortcut for the kron operation

[dandanua]

Hello!

I think that the symbol ⊗ is quite a common notation for denoting tensor products and the Kronecker product of matrices in linear algebra. I always use ⊗ = kron in my code and it works without any problems (in its infix form A⊗B). Are there any reasons why this could not be included by default in the LinearAlgebra.jl?

[stevengj]

(Note that you should use const ⊗ = kron to make sure it is constant.)

I guess the concern is that this symbol is used for other things in math too. But in the context of LinearAlgebra I agree that it might be reasonable.

[stevengj]

For example ⊗ is also defined in QuantumSymbolics.jl, ParametricOperators.jl, TensorOperator.jl, TensorRefinement.jl, TensorKit.jl, AbstractTensors.jl, SciMLOperators.jl, AtomicLevels.jl, AeroFuse.jl, KroneckerProducts.jl, NumericalRepresentationTheory.jl, DoubleFloats.jl, QuantumClifford.jl, NDTensors.jl, Oscar.jl, QuantumCumulants.jl, QuantumLattices.jl, ThreeBodyDecays.jl, BEAST.jl, FinitePosets.jl, MonteCarloMeasurements.jl, SpecialPolynomials.jl, Grassman.jl, LinearMaps.jl, HalfEdges.jl, AbstractPDEInterfaces.jl, Equations.jl, AeroMDAO.jl, DoloYAML.jl, NodalPolynomialSpaces.jl, FourierSpaces.jl, UniqueKronecker.jl, Kronecker.jl, TotalLeastSquares.jl, QSymbolicsBase.jl, RandomizedLinearAlg.jl, ExactDiagonalization.jl, SimpleTropical.jl, ApproxFunBase.jl, ColorVectorSpace.jl, EtherSPH.jl, MeasureBase.jl, AlgebraicRewriting.jl, and others…

Many of these would then need to decide if they want to shadow the new LinearAlgebra definition (the default) or import/extend it. And there might be side effects we aren't considering, if they use kron to mean something different.

[dandanua]

It could be confusing if someone needs to use both the LinearAlgebra.jl and a package where ⊗ is defined with some other meaning, but I don't think it could break anything. I can't imagine what kind of side affects could appear.

Yes, I define ⊗ with the const (learned that the hard way).

[mcabbott]

Xref JuliaLang/julia#35150, which defined this symbol for (A ⊗ B)[i,j,k,l,m] == A[i,j] * B[k,l,m] (for any ndims). But was reverted & moved to TensorCore.jl.

That's a more standard definition for this symbol than Base.kron, which reshapes & has its indices in a less natural order: vec(a * transpose(b)) ≈ kron(b, a) for vectors.

[araujoms]

Well I'm glad that PR got reverted, I think making ⊗ mean anything other than kron is asking for trouble. Sadly the comments in that PR show that there's no consensus about what ⊗ should mean, and therefore we'll never get it into LinearAlgebra.

Which is double frustrating, because I'd love to have it in LinearAlgebra precisely so that I could use it in my package without confusing users that might expect it to mean something else.

[stevengj]

That's a more standard definition for this symbol than Base.kron, which reshapes & has its indices in a less natural order: vec(a * transpose(b)) ≈ kron(b, a) for vectors.

On matrices, Base.kron implements a widely used and longstanding definition for the matrix Kronecker product. So I wouldn't say it is "less standard". Indeed, the matrix definition we are using dates back to the 19th century AFAIK, and I'm guessing it precedes the "tensor" version, though I don't know who used $\otimes$ first.

But I agree that it is unfortunate that the matrix version swaps the ordering compared to the multilinear-algebra "tensor product" version.

[dandanua]

Xref JuliaLang/julia#35150, which defined this symbol for (A ⊗ B)[i,j,k,l,m] == A[i,j] * B[k,l,m] (for any ndims). But was reverted & moved to TensorCore.jl.

That's a more standard definition for this symbol than Base.kron, which reshapes & has its indices in a less natural order: vec(a * transpose(b)) ≈ kron(b, a) for vectors.

This issue is akin to the big-endian vs little-endian convention (BTW, I've tried to make parallels between this and tensor products in this short doc about conventions in math). I find little-endian convention to be more natural, but big-endian is simply much more common and older convention. Same with the Kronecker product, it's older and used more widely.

[araujoms]

I think what is swapping the order is the col-major orientation. If we define a row-based vectorization rvec(a) = vec(transpose(a)) then rvec(a * transpose(b)) ≈ kron(a, b). But like one-based indexing this is a design flaw that cannot be fixed.

[dandanua]

Apart from the row- vs column-based vectorization, there are other ways to get a swapped kron: redefine the matrix product to be A * B := B * A, or redefine any matrix to be M := transpose(M), or redefine where we put the first argument in functions - on the left or right, etc. The doc I've linked regards a wider picture. It's all about conventions, there is no "right" or "wrong" solution. However, the definition of the Kronecker product is precise, and I don't think it's Julia purpose to redefine it.

Anyway, the issue of what is better, kron(a,b) or the swapped variant is not directly related to the shortcut question. I don't think it makes sense to define ⊗(a,b) = kron(b,a) in LinearAlgebra.jl. As for the multilinear definition, I think it was a wise decision to move it into a dedicated package.

[stevengj]

@araujoms, note that numpy.kron uses the same definition as Julia's (which definition, again, is 150+ years old) even though it defaults to row-major storage. And the notation vec(A) for stacking the columns of A goes back to 1950 at least, while the identity $\mathrm{vec}(ABC) = (C^T \otimes A) \mathrm{vec}(B)$ apparently goes back even earlier to 1935 ("Roth's column lemma").

I agree that it would be nicer (in hindsight) the other way around, but you cannot argue that it is nonstandard, or that it is an idiosyncratic choice specific to Julia.

[araujoms]

Python's vec (called ravel) is row major, and thus is consistent with its kron. I'm only saying that Julia's choices for kron and vec are incompatible with each other. FWIW I think Julia made the right call with kron and the wrong one with vec. Your source also notes that the row-major version of vec is often used.

I much prefer the identity $\mathrm{vec}(ABC) = (A \otimes C^T) \mathrm{vec}(B)$, no swapping the order between $A$ and $C$.

[stevengj]

Julia's choices for kron and vec are incompatible with each other

Like them or not, they are not merely "Julia's" choices … they are both the standard meanings of these terms in linear algebra. The source notes that some authors historically used a row-major vec, but it is not the most common definition, and was typically given a different name.

Personally, I would also have preferred row-major storage, but anyway … water under the bridge at this point.