Using a constructor inside inside a lambda, with a specified wider return type, causes a spooky java.lang.NoClassDefFoundError error in the outermost object's class initialization

[neko-kai]

Compiler version

3.7.2 and 3.8.0-RC1-bin-20250814-0cf7a18-NIGHTLY

Minimized code

To minimize we have to run the following code without zio on the classpath. We may do that by executing the following, assuming you have Coursier installed as cs on your system:

mkdir target
cs launch scala-cli:1.9.0 -- example.scala -d target/
java -cp target:`cs fetch org.scala-lang:scala-library:2.13.6` zio.main

Where example.scala:

//> using scala 3.7.2
//> using dep dev.zio::zio::2.1.21
package zio

object main extends App {
  println(outer.helloWorld)
}

object outer {
  def helloWorld = "Hello world!"

  object outer1 {
    object inner {
      def crashCausingLambdaFn: Boolean => ZIO[Any, Nothing, Boolean] = (f: Boolean) => new ZIO.Sync(zio.Trace.empty, () => f) // if this line is commented out, the code works.
      def manualCastMakesItWorkSomehow: Boolean => ZIO[Any, Nothing, Boolean] = 
        (f: Boolean) => new ZIO.Sync(zio.Trace.empty, () => f).asInstanceOf[ZIO[Any, Nothing, Boolean]]
      def applyMethodWorks: Boolean => ZIO[Any, Nothing, Boolean] =
        (a: Boolean) => { val x = ZIO.Sync.apply[Boolean]; x(zio.Trace.empty, () => a) }
    }
  }
}

EDIT: I minimized the example further and removed the inlines that weren't the real cause of this behavior following @som-snytt's comment below.

Output

Exception in thread "main" java.lang.NoClassDefFoundError: zio/ZIO
	at zio.main$.<clinit>(example.scala:9)
	at zio.main.main(example.scala)
Caused by: java.lang.ClassNotFoundException: zio.ZIO
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:528)
	... 2 more

Expectation

Expected to print Hello World.

• If the line with crashCausingLambdaFn is commented out, the code works.
• If we apply a manual cast to ZIO - the code works, the constructor call by itself does not cause the crash, only when combined with implicit widening.
• If we forcefully call ZIO.Sync.apply instead of new, the code works.
• Weirdly enough, the code also works if the return type signature is omitted in the inline definition (succeedNewNoTypeSig in example) That was caused by a narrower type being inferred.

Note: the reason to run code with some of the dependencies missing is to be able to ship libraries with batteries-included support for various frameworks - e.g. with typeclass instances and conversions for cats & ZIO, without the downstream user necessarily having to use them, by compiling with said libraries as % Optional dependencies.

[som-snytt]

IIRC, it's at the discretion of the JVM how eagerly to load classes on class initialization.

The difference here is whether the lifted method implementing the function returns ZIO or ZIO$Sync (an inner class).

It works if succeedNew is transparent inline (which fixes the signature) or if it is not inline (which presumably makes the JVM less eager to load its result class; perhaps because it's no longer an invokedynamic but just an invokevirtual).

This sort of problem used to come up with frameworks relying on loading and configuring during class init, for just the same reason, that it forced loading of classes not actually present.

Edit: maybe just the text in 5.4 Linking of the JVMS applies.

[neko-kai]

@som-snytt
Indeed, this seems to be a function of return type + constructor call (applyMethodWorks works even if annotated).
The crash reproduces with the same code inlined manually, but with an added type annotation:

def crashCausingLambdaFn: Boolean => ZIO[Any, Nothing, Boolean] =
  (f: Boolean) => new ZIO.Sync(zio.Trace.empty, () => f)

def applyMethodWorks: Boolean => ZIO[Any, Nothing, Boolean] =
  (a: Boolean) => { val x = ZIO.Sync.apply[Boolean]; x(zio.Trace.empty, () => a) }

But... why? The type parameter of Function1 is erased, why is it affecting the output bytecode so much and only specifically when a constructor call is involved?

There's also a moment that the error wouldn't happen if lambdas were lifted to their nearest owner, as in Scala 2, instead of the outermost possible owner. Then the lifted methods would be defined inside outer$outer1$inner$.class and not cause a crash when only outer$ is involved.

[neko-kai]

@som-snytt
On a second look, this seems to not be solely a function of a wider return type in the lambda + constructor call. If I apply a cast to ZIO manually, the crash doesn't happen. Only implicit widening causes this.

def manualCastMakesItWorkSomehow: Boolean => ZIO[Any, Nothing, Boolean] = 
  (f: Boolean) => { new ZIO.Sync(zio.Trace.empty, () => f).asInstanceOf[ZIO[Any, Nothing, Boolean]] }

[som-snytt]

@neko-kai Maybe the checkcast introduced in your last example will always force loading of ZIO, so that it doesn't have to happen eagerly. That is only a surmise. JVMS language is that the error should be thrown when "some action is taken that might (directly or indirectly) require linkage; or when it uses the symbolic reference".

Other sample "workaround" to control lifting:

  object zouter1 {
    class inner {
      def env = false
      def workaround: Boolean => ZIO[Any, Nothing, Boolean] =
          (f: Boolean) =>
            if env then ???
            new ZIO.Sync(zio.Trace.empty, () => f)
    }
    val inner = new inner
  }