Stackoverflow in typechecker.Typers$Typer in scala 2.12.3/2.12.4

[fanf]

I'm observing a stackoverflow in typechecker with ScalaIDE 4.7.0 when using scalac 2.12.3 or 2.12.4, which is not present with scala 2.12.2. The stackoverflow does not happen when I'm compiling from the command line using maven.
I saw a lot of variation of the problem (call site initialization exception and java.lang.UnsupportedOperationException: Position.point on NoPosition) but these other problem seems to be consequences of the stackoverflow.

Of course I tried to set a bigger Xss, but it does not change anything (even with 256m), and the fact that the maven compilation works or that it works on 2.12.2 with normal Xss tends to let me think that it is a genuine bug.

I don't have at all a minimal reproducer, and the stacktraces are not very telling, but the problem happen with an open source project: https://github.com/Normation/rudder/ (on branch master).

It may or not may be related to Daniel "infinite macro expansion in scala 2.12.3/2.12.4" #10584 or perhaps both have same root cause (rudder-core uses doobie and so macro expansion).

Nonetheless, here are the SOE and other exception I saw:

2017-10-26 18:33:47,710 ERROR [main] - org.scala-ide.sdt.core - org.scala-ide.sdt.core - org.scala-ide.sdt.core - 0 - Error in Scala compiler
java.lang.StackOverflowError
    at scala.tools.nsc.typechecker.Contexts$Context.make(Contexts.scala:463)
    at scala.tools.nsc.typechecker.Contexts$Context.makeNewScope(Contexts.scala:516)
    at scala.tools.nsc.typechecker.Typers$Typer.typedOutsidePatternMode$1(Typers.scala:5511)
    at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5546)
    at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5553)
    at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5589)
    at scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5619)
    at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5563)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5567)
    at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5670)
    at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5676)
    at scala.tools.nsc.typechecker.Typers$Typer.typedSelectOrSuperCall$1(Typers.scala:4999)
    at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5537)
    at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5553)
    at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5589)
    at scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5619)
    at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5563)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5567)
    at scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$38(Typers.scala:4697)
    at scala.tools.nsc.typechecker.Typers$Typer.silent(Typers.scala:699)
    at scala.tools.nsc.typechecker.Typers$Typer.normalTypedApply$1(Typers.scala:4699)
    at scala.tools.nsc.typechecker.Typers$Typer.typedApply$1(Typers.scala:4746)
    at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5536)
    at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5553)
    at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5589)
    at scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5619)
    at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5563)
   .... and on and on...

[fanf]

It may, perhaps, not sure anymore at anything, that the problem might be link with the use of the "-verbose" option. Removing that flag seems to make it disapear (but as I don't have a clean way to trigger it in the first place, I'm not sure of anything).

[tpolecat]

Yeah we're seeing java.lang.UnsupportedOperationException: Position.point on NoPosition.

[olafurpg]

Upgrading from 2.12.3 to 2.12.4 introduced a StackOverflowError in inductive shapeless derivation in the scalafix repo scalacenter/scalafix#477 (comment) Increasing -Xss8m fixed the problem. This regression took a while to track down since the SO was caught by the compiler during implicit expansion and only reported under -Xlog-implicits. I'm not sure if it's directly related to this PR, but wanted to leave a note somewhere. At least, I'm not the only one hitting on SO while upgrading in 2.12.x

[olafurpg]

I took the liberty to open another issues about the fact that the SO was getting swallowed and not reported without -Xlog-implicits #10649

[fanf]

It may be related to #10552

[fanf]

I've a variant of that one on IntelliJ (I believe I also got it from time to time on scala IDE) with a NPE:

Error:scala: ## Exception when compiling 55 sources to /home/fanf/java/workspaces/rudder-project/rudder/rudder-core/target/test-classes
null
scala.tools.nsc.typechecker.Typers$Typer.typedBlock(Typers.scala:2486)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$96(Typers.scala:5505)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typerWithLocalContext$1(Typers.scala:486)
scala.tools.nsc.typechecker.Typers$Typer.typedOutsidePatternMode$1(Typers.scala:486)
scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5540)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5547)
scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5584)
scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5616)
scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5557)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5562)
scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5667)
scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5673)
scala.tools.nsc.typechecker.Typers$Typer.typedSelectOrSuperCall$1(Typers.scala:5008)
scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5531)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5547)
scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5584)
scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5616)
scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5557)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5562)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$38(Typers.scala:4708)
scala.tools.nsc.typechecker.Typers$Typer.silent(Typers.scala:698)
scala.tools.nsc.typechecker.Typers$Typer.normalTypedApply$1(Typers.scala:4710)
scala.tools.nsc.typechecker.Typers$Typer.typedApply$1(Typers.scala:4757)
scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5530)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5547)
scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5584)
scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5616)
scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5557)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5562)
scala.tools.nsc.typechecker.Typers$Typer.typedBlock(Typers.scala:2457)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$96(Typers.scala:5505)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typerWithLocalContext$1(Typers.scala:486)
scala.tools.nsc.typechecker.Typers$Typer.typedOutsidePatternMode$1(Typers.scala:486)
scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5540)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5547)
scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5584)
scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5616)

...... and again and again and again

[thestuckster]

Is this issue being actively worked on by anyone? We suddenly started getting the same problem on my project at work. We bumped the stack size which has fixed the issue but, that seems more like a band-aid rather than an actual fix.

We are seeing this issue when compiling with Maven on the command line instead of just the IDE.

[joan38]

Same here with 2.12.4

[normana400]

bump

[normana400]

fwiw, Im having this issue w/ shapeless / slickless in Scala 2.11.11

[SethTisue]

some of y'all know this already, but maybe not everyone does: it's considered normal in Scala that you may have to increase JVM stack size in order to compile some kinds of especially involved or nested Scala code.

does anyone know a way to reproduce the problem they're having on Scala 2.12.5 (yes, I mean 2.12.5 specifically, now that it's out), with a minimal example and without needing access to your project's entire source tree? without that, it's really hard to know whether this ticket even identifies a specific issue, or whether it's just a miscellaneous collection of people reporting a miscellaneous collection of StackOverflowErrors in miscellaneous versions of Scala, which could have miscellaneous causes.

(I'm not criticizing anyone who's reported or commented here, but in order to actually do anything about this, we need to converge on something more specific and actionable.)

note that 2.12.5 fixes #10552, so trying again in 2.12.5 may produce different error logs that may continue additional clues that we didn't have before.

[rafalmag]

I had the same issue and I can confirm that increasing of the stack size from default one (1MB to 5MB) helped.