Specialize Diagonal * Adjoint

[jishnub]

This reinstates slightly altered versions of the methods that were removed in JuliaLang/julia#52389. Sort of fixes #1205, although this doesn't recover the full performance. However, this version is more general, and works with the example presented in JuliaLang/julia#52389. There's still a performance regression, but the full performance may only be obtained for mutable matrices, and we may not assume mutability in general.

Performance:
v1.10:

julia> n = 100
100

julia> A = adjoint(sparse(Float64, I, n, n));

julia> B = Diagonal(ones(n));

julia> @btime $A * $B;
  837.119 ns (5 allocations: 2.59 KiB)

This PR

julia> @btime $A * $B;
  1.106 μs (15 allocations: 5.56 KiB)

We need double the allocations here compared to earlier, as we firstly materialize D' * A', and then we again copy the adjoint of this result. I wonder if this may be reduced.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.93%. Comparing base (5cf41c4) to head (cf94706).
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1207   +/-   ##
=======================================
  Coverage   91.92%   91.93%           
=======================================
  Files          34       34           
  Lines       15372    15384   +12     
=======================================
+ Hits        14131    14143   +12     
  Misses       1241     1241           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dkarrasch]

Should we add methods for strided AdjOrTrans for even better performance? But that doesn't guarantee mutability of individual entries, right? Otherwise, looks good to go.

[jishnub]

We may add these methods, but non-strided mutable matrices (such as SparseArrays) will still be slow. Perhaps we need a mutability trait, like ArrayInterface.

[dkarrasch]

SparseArrays) will still be slow

We could take care of that case specifically over there. But performance is not too bad for that case, as you showed.

[dkarrasch]

LGTM. Could you perhaps take a look at SparseArrays.jl to see what it needs there to bring back the inplace multiplication?

[jishnub]

For the future: if #1210 is merged, we may specialize mul directly, instead of redirecting * to _diag_adj_mul.

[dkarrasch]

For the future: if #1210 is merged, we may specialize mul directly, instead of redirecting * to _diag_adj_mul.

I'd suggest we do that: having only generic * methods, and everything else (like bidiag*hessenberg, and whatnot) should be dispatched through mul.

[dkarrasch]

I forgot: if we want to backport, we can't use the new stuff. So mul has to go into a new PR.