Immutable tensors

[pygy]

As proposed in this post.

Having immutable tensors would elegantly solve the temporary allocation problem that currently plagues vectorized code in Julia.

Evaluation would occur lazily in a devectorized fashion when mandatory, and the result would the be cached in the object, an immutable type whose "realized" field is a one slot array. That way you get the best of both worlds: immutability for operations and a writable field for the result.

I know that Stefan doesn't like memoization, but I think it could make sense in this case.

Beside transposition (#6837), addition, subtraction, scalar operations and maybe other ones could be trivially implemented without any temporary allocation.

Not sure if there's a way to make it fast, though. It may be possible to analyze the code flow (along with type analysis?) and prepare specialized code inlined at the evaluation point.

Proposed syntax:

matrix = [| 1, 2 |
          | 3, 4 |]

inline = [|1, 2| |3, 4|]

alternatively [|1, 2; 3, 4|]

threedee = [|| 1, 2 | | 5, 6 ||
            || 3, 4 | | 7, 8 ||]

xyplane = [|| 1, 2 ||
           || 3, 4 ||]

xzplane = [|| 1, 2 | | 5, 6||]

yzplane = [|| 1 | | 5 ||
           || 3 | | 7 ||]

alt3d = [|| 1, 2 ; 5, 6 ;; 3, 4 ; 7, 8 ||]

And let's go wild:

fourdee = [||| 1, 2 | | 5, 6 ||
            || 3, 4 | | 7, 8 |||

           ||| a, b | | e, f ||
            || c, d | | g, h |||]

Parsing that may not beautiful ;-)

Is there a language out there that allows to cleanly represent 4d data?

---

Edit:

I know that there's an inconsistency between the 2d representation and the other two:

For matrices, , is the x separator and | the y one.

In 3d and 4d, it's x => ,, y => ||, z => | and w => |||.

It allows to get a nice 2d view along these lines:

+----------z-------->
|  +--x-->
|  |
|  y
|  |
|  V
|
w
|
V

... rather than this which you would get with symbolically consistent separators.

+----------y-------->
|  +--x-->
|  |
|  z
|  |
|  V
|
w
|
V

It may take some time to get past the cognitive dissonance, but I think it's worth it for the visual result.

[tknopp]

This sounds very similar to #5857. I am not sure if for immutable small arrays temporaries are an issue because everything would be hold in registers, no?

[pygy]

This about eliminating the allocation of large temporary arrays by delaying the computation until needed.

It is insnpired by this library by @dpo

https://github.com/dpo/linop.jl

julia> A = 2 .* [|1, 2| #imagine it is large :-)
                 |3, 4|]
MatrixScalarProduct # <: AbstractMatrix
  n: 2
  M: [|1, 2; 3, 4|]
  realized: {nothing}

julia> B = A'
MatrixTransposition # <: AbstractMatrix
  M: A
  realized: {nothing}

julia> B = B + [|5, 6|
                |7, 8|]
MatrixSum # <: AbstractMatrix
M1: B
M2: [|5, 6; 7, 8|]
realized: {nothing}

julia> print(C[1,1])
# compute C here with no temporary allocation, 
# store the result then get the element.
7
# now C.realized == {[|7, 12; 11, 16|]}

Now that I think of it, immutables may not be ideal for the intermediate steps, since, AFAIK, they hold the full data, which means that a copies of the source matrices would be necessary. An immutable reference would be needed for efficiency.

[JeffBezanson]

I would like to have syntax for n-d arrays. Using multiple | is not a bad proposal. It is always annoying to have data structures without syntax.

[JeffBezanson]

Although I should add that this is almost impossible to distinguish from uses of the | operator, on the closing end. We might need to use |, or an immediately following newline, or something similar.

[pygy]

"value—pipe—space—pipe—value" is currently invalid syntax. You could look for | | before | in that context... or mandate the ; syntax for inline definitions.

Edit: note that, for dimensions higher than 4, the | / || inversion described in the edit of the first message becomes a liability. Are there use cases for (5+)d data literals?

[KristofferC]

This seems very much as a thing for a package and less so for base.