Removing the magic from dotty.runtime.Arrays

[sjrd]

dotty.runtime.Arrays has a bunch of magic methods, including the very magic newRefArray, which is one of the 3 methods that stay generic after erasure.

It is very easy to get rid of this complexity, by given an actual parameter of type Class[T] to newRefArray:

def newRefArray[T <: AnyRef](componentType: Class[T], length: Int): Array[T]

The back-end still needs to know about this magic method, and "assume" that componentType is a Literal(Constant(tpe)) (i.e., crash if it isn't). But then the method does not need to be generic anymore after erasure, which is a substantial win, IMO.

We can remove even more magic by given actual bodies to all the newXArray method, leveraging java.lang.reflect.Array.newInstance:

def newRefArray[T <: AnyRef](componentType: Class[T], length: Int): Array[T] =
  java.lang.reflect.Array.newInstance(componentType, length).asInstanceOf[Array[T]]

def newByteArray(length: Int): Array[Byte] =
  java.lang.reflect.Array.newInstance(classOf[Byte], length).asInstanceOf[Array[Byte]]

...

def newUnitArray(length: Int): Array[Unit] =
  java.lang.reflect.Array.newInstance(classOf[BoxedUnit], length).asInstanceOf[Array[Unit]]

and now the back-end does not need to treat these methods specially. Their user-space implementation is perfectly correct.

Of course, on the JVM, that implementation is inefficient, so the JVM back-end would still specialize them. But other back-ends need not have the same problem. For example, Scala.js does not need to specialize those to get full speed, because Array.newInstance is intrisified by the optimizer when the componentType is a literal classOf.

[DarkDimius]

@sjrd I was proposing a similar change at one point. Btw, newByteArray does not need to use reflection, it could simply call new Array[Byte](length). Same with other primitives.

I do not remember why this proposal was rejected.

[DarkDimius]

Btw, newByteArray does not need to use reflection, it could simply call new ArrayByte

but needs to implemented in java. Otherwise we would rewrite it to recursive method).

Thinking about it, I was proposing even a bigger change.
The other 2 generic methods after erasure are isInstanceOf and asInstanceOf.

I was proposing such rewriting to make them not be generic:

1. For every class X, generate a symbol $isInstanceOf in companion of X.
2. rewrite expr.isInstanceOf[X] to X$.$isInstanceOf(expr).

Similarly, backends would still need to know about this change, but then we would have no generic methods after erasure.

TBH, I do not think that eliminating 1 generic method out of 3 is worth it, as transformations after erasure would still need to be handle TypeApply. But if we can rewrite all 3 methods: newRefArray, asInstanceOf and isInstanceOf than phases after erasure would not need to handle TypeApply at all.

[sjrd]

but needs to implemented in java. Otherwise we would rewrite it to recursive method).

Yep, but then we have to re-implement them in Scala anyway for other back-ends.

Thinking about it, I was proposing even a bigger change.

That might be more controversial, indeed. At least, for newRefArray, there is a correct user-space implementation in the non-generic case.

That is not true for asInstanceOf[T]: there does not exist any user-space implementation of it, even with one clone per class. isInstanceOf[T] can actually be implemented in a similar way as newRefArray using reflection, without even one per class:

def isInstanceOfRef(tpe: Class[_], obj: AnyRef): Boolean =
  tpe.isInstance(obj)

Note that this version also deals with isInstanceOf[Array[X]], unlike the one-clone-per-class version that you suggest.

asInstanceOf does not have a user-space implementation, and that is what sets it apart. In my book, it would deserve an actual Tree because of that!

[sjrd]

That said, I think it's worth "fixing" the newXArray situation even if we keep is/asInstanceOf as they are, because that brings us back to the status quo with scalac. Right now the situation is worse than in scalac. And newXArray should be non-controversial, I think.

[odersky]

I agree it would be good to try that change.