Build and publish against current Scala 2.12 milestone

[JoshRosen]

It would be great if jackson-module-scala could build and publish artifacts against the current Scala 2.12 milestone (currently 2.12.0-M4). This is a blocker for building Apache Spark against Scala 2.12 (see SPARK-14222).

[JoshRosen]

Note: now that 2.12.0-M4 is out we should compile against that and not M3 because apparently there are some binary incompatibilities between the two milestones.

[cowtowncoder]

@JoshRosen contributions would be most welcome -- Scala module has lots of work, but limited developer resources. I don't know how much work new backend would be, although I have hard 2.12 is supposed to be easier to migrate to than earlier versions.

[JoshRosen]

I think that Spark's use of jackson-module-scala is fairly shallow, so from my perspective it might be less effort for us to just remove this dependency from Spark. I'm exploring this option at apache/spark#12213 and if that works then I'll close this request.

[cowtowncoder]

@JoshRosen yes, it may make sense to remove jackson scala module if so (fwtw basic jackson-databind may work well enough even from Scala). Either way, let's keep this open: I am sure 2.12 version will be needed by some users so it's good to have placeholder.

[nbauernfeind]

I will release an M4 version of 2.7.3 in the very near future. Is 2.7.x sufficient or do you need older versions as well? To keep this up to date, can you please remember to ping us as scala 2.12 releases new milestones? New releases of the scala compiler are unfortunately no longer a day-to-day requirement for me.

[nbauernfeind]

Hmm, I'm having trouble compiling actually. Have you seen this before?

[info] 'compiler-interface' not yet compiled for Scala 2.12.0-M4. Compiling...
java.lang.VerifyError: Uninitialized object exists on backward branch 209
Exception Details:
  Location:
    scala/collection/immutable/HashMap$HashTrieMap.split()Lscala/collection/immutable/Seq; @249: goto
  Reason:
    Error exists in the bytecode
  Bytecode:
    0x0000000: 2ab6 005c 04a0 001e b200 b4b2 00b9 04bd
    0x0000010: 0002 5903 2a53 c000 bbb6 00bf b600 c3c0
    0x0000020: 00c5 b02a b600 36b8 0040 3c1b 04a4 015e
    0x0000030: 1b05 6c3d 2a1b 056c 2ab6 0036 b700 c73e
    0x0000040: 2ab6 0036 021d 787e 3604 2ab6 0036 0210
    0x0000050: 201d 647c 7e36 05bb 0019 59b2 00b9 2ab6
    0x0000060: 0038 c000 bbb6 00cb b700 ce1c b600 d23a
    0x0000070: 0619 06c6 001a 1906 b600 d6c0 0082 3a07
    0x0000080: 1906 b600 d9c0 0082 3a08 a700 0dbb 00db
    0x0000090: 5919 06b7 00de bf19 073a 0919 083a 0abb
    0x00000a0: 0002 5915 0419 09bb 0019 59b2 00b9 1909
    0x00000b0: c000 bbb6 00cb b700 ce03 b800 e43a 0e3a
    0x00000c0: 0d03 190d b900 e801 0019 0e3a 1136 1036
    0x00000d0: 0f15 0f15 109f 0027 150f 0460 1510 190d
    0x00000e0: 150f b900 eb02 00c0 0005 3a17 1911 1917
    0x00000f0: b800 ef3a 1136 1036 0fa7 ffd8 1911 b800
    0x0000100: f3b7 0061 3a0b bb00 0259 1505 190a bb00
    0x0000110: 1959 b200 b919 0ac0 00bb b600 cbb7 00ce
    0x0000120: 03b8 00e4 3a13 3a12 0319 12b9 00e8 0100
    0x0000130: 1913 3a16 3615 3614 1514 1515 9f00 2715
    0x0000140: 1404 6015 1519 1215 14b9 00eb 0200 c000
    0x0000150: 053a 1819 1619 18b8 00f6 3a16 3615 3614
    0x0000160: a7ff d819 16b8 00f3 b700 613a 0cb2 00fb
    0x0000170: b200 b905 bd00 0259 0319 0b53 5904 190c
    0x0000180: 53c0 00bb b600 bfb6 00fe b02a b600 3803
    0x0000190: 32b6 0100 b0                           
  Stackmap Table:
    same_frame(@35)
    full_frame(@141,{Object[#2],Integer,Integer,Integer,Integer,Integer,Object[#110]},{})
    append_frame(@151,Object[#130],Object[#130])
    full_frame(@209,{Object[#2],Integer,Integer,Integer,Integer,Integer,Object[#110],Object[#130],Object[#130],Object[#130],Object[#130],Top,Top,Object[#25],Object[#60],Integer,Integer,Object[#108]},{Uninitialized[#159],Uninitialized[#159],Integer,Object[#130]})
    full_frame(@252,{Object[#2],Integer,Integer,Integer,Integer,Integer,Object[#110],Object[#130],Object[#130],Object[#130],Object[#130],Top,Top,Object[#25],Object[#60],Integer,Integer,Object[#108]},{Uninitialized[#159],Uninitialized[#159],Integer,Object[#130]})
    full_frame(@312,{Object[#2],Integer,Integer,Integer,Integer,Integer,Object[#110],Object[#130],Object[#130],Object[#130],Object[#130],Object[#2],Top,Object[#25],Object[#60],Integer,Integer,Object[#108],Object[#25],Object[#60],Integer,Integer,Object[#108]},{Uninitialized[#262],Uninitialized[#262],Integer,Object[#130]})
    full_frame(@355,{Object[#2],Integer,Integer,Integer,Integer,Integer,Object[#110],Object[#130],Object[#130],Object[#130],Object[#130],Object[#2],Top,Object[#25],Object[#60],Integer,Integer,Object[#108],Object[#25],Object[#60],Integer,Integer,Object[#108]},{Uninitialized[#262],Uninitialized[#262],Integer,Object[#130]})
    full_frame(@395,{Object[#2],Integer},{})

    at scala.collection.immutable.HashMap$.scala$collection$immutable$HashMap$$makeHashTrieMap(HashMap.scala:179)
    at scala.collection.immutable.HashMap$.scala$collection$immutable$HashMap$$makeHashTrieMap(HashMap.scala:183)
    at scala.collection.immutable.HashMap$HashMap1.updated0(HashMap.scala:211)
    at scala.collection.immutable.HashMap.$plus(HashMap.scala:59)
    at scala.collection.immutable.HashMap.$plus(HashMap.scala:62)
    at scala.collection.immutable.Map$Map4.updated(Map.scala:201)
    at scala.collection.immutable.Map$Map4.$plus(Map.scala:202)
    at scala.collection.immutable.Map$Map4.$plus(Map.scala:180)
    at scala.collection.mutable.MapBuilder.$plus$eq(MapBuilder.scala:29)
    at scala.collection.mutable.MapBuilder.$plus$eq(MapBuilder.scala:25)
    at scala.collection.generic.Growable.scala$collection$generic$Growable$$$anonfun$1(Growable.scala:59)
    at scala.collection.generic.Growable$$Lambda$19/1633498752.apply(Unknown Source)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
    at scala.collection.generic.Growable.$plus$plus$eq(Growable.scala:59)
    at scala.collection.mutable.MapBuilder.$plus$plus$eq(MapBuilder.scala:25)
    at scala.collection.generic.GenMapFactory.apply(GenMapFactory.scala:48)
    at scala.sys.package$.env(package.scala:61)
    at scala.tools.nsc.settings.ScalaSettings.defaultClasspath(ScalaSettings.scala:29)
    at scala.tools.nsc.settings.MutableSettings.defaultClasspath(MutableSettings.scala:19)
    at scala.tools.nsc.settings.ScalaSettings.$init$(ScalaSettings.scala:59)
    at scala.tools.nsc.settings.MutableSettings.<init>(MutableSettings.scala:20)
    at scala.tools.nsc.Settings.<init>(Settings.scala:12)
    at scala.tools.nsc.Driver.process(Driver.scala:41)
    at scala.tools.nsc.Main.process(Main.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at sbt.compiler.RawCompiler.apply(RawCompiler.scala:26)
    at sbt.compiler.AnalyzingCompiler$$anonfun$compileSources$1$$anonfun$apply$2.apply(AnalyzingCompiler.scala:146)
    at sbt.compiler.AnalyzingCompiler$$anonfun$compileSources$1$$anonfun$apply$2.apply(AnalyzingCompiler.scala:142)
    at sbt.IO$.withTemporaryDirectory(IO.scala:291)
    at sbt.compiler.AnalyzingCompiler$$anonfun$compileSources$1.apply(AnalyzingCompiler.scala:142)
    at sbt.compiler.AnalyzingCompiler$$anonfun$compileSources$1.apply(AnalyzingCompiler.scala:139)
    at sbt.IO$.withTemporaryDirectory(IO.scala:291)
    at sbt.compiler.AnalyzingCompiler$.compileSources(AnalyzingCompiler.scala:139)
    at sbt.compiler.ComponentCompiler$$anonfun$compileAndInstall$1.apply(ComponentCompiler.scala:63)
    at sbt.compiler.ComponentCompiler$$anonfun$compileAndInstall$1.apply(ComponentCompiler.scala:60)
    at sbt.IO$.withTemporaryDirectory(IO.scala:291)
    at sbt.compiler.ComponentCompiler.compileAndInstall(ComponentCompiler.scala:60)
    at sbt.compiler.ComponentCompiler$$anonfun$getLocallyCompiled$1.apply$mcV$sp(ComponentCompiler.scala:50)
    at sbt.IfMissing$Define.apply(ComponentManager.scala:75)
    at sbt.ComponentManager.sbt$ComponentManager$$createAndCache$1(ComponentManager.scala:39)
    at sbt.ComponentManager$$anonfun$sbt$ComponentManager$$fromGlobal$1$1.apply(ComponentManager.scala:27)
    at sbt.ComponentManager$$anonfun$sbt$ComponentManager$$fromGlobal$1$1.apply(ComponentManager.scala:26)
    at sbt.ComponentManager$$anon$1.call(ComponentManager.scala:50)
    at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
    at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
    at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
    at xsbt.boot.Using$.withResource(Using.scala:10)
    at xsbt.boot.Using$.apply(Using.scala:9)
    at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
    at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
    at xsbt.boot.Locks$.apply0(Locks.scala:31)
    at xsbt.boot.Locks$.apply(Locks.scala:28)
    at sbt.ComponentManager.lock(ComponentManager.scala:50)
    at sbt.ComponentManager.lockGlobalCache(ComponentManager.scala:49)
    at sbt.ComponentManager.sbt$ComponentManager$$fromGlobal$1(ComponentManager.scala:25)
    at sbt.ComponentManager$$anonfun$files$1$$anonfun$apply$2.apply(ComponentManager.scala:44)
    at sbt.ComponentManager$$anonfun$files$1$$anonfun$apply$2.apply(ComponentManager.scala:44)
    at sbt.ComponentManager.sbt$ComponentManager$$getOrElse$1(ComponentManager.scala:32)
    at sbt.ComponentManager$$anonfun$files$1.apply(ComponentManager.scala:44)
    at sbt.ComponentManager$$anonfun$files$1.apply(ComponentManager.scala:44)
    at sbt.ComponentManager$$anon$1.call(ComponentManager.scala:50)
    at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
    at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
    at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
    at xsbt.boot.Using$.withResource(Using.scala:10)
    at xsbt.boot.Using$.apply(Using.scala:9)
    at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
    at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
    at xsbt.boot.Locks$.apply0(Locks.scala:31)
    at xsbt.boot.Locks$.apply(Locks.scala:28)
... etc

[JoshRosen]

@nbauernfeind, are you using a Java 8 JDK? Scala 2.12 requires JDK 8+.

[nbauernfeind]

I believe so... but maybe I'm missing something more obvious:

$ java -version
java version "1.8.0_40"
Java(TM) SE Runtime Environment (build 1.8.0_40-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
$ echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home

[JoshRosen]

Which SBT version are you using? I've noticed that this project doesn't seem to have a project/build.properties file for fixing the SBT version, so there's a chance we're using different versions.

[cowtowncoder]

@nbauernfeind As to 2.7.3 vs older versions: the only one missing is 2.6.6 which would be nice to have, but not absolutely essential. It is the last patch release for 2.6, so there won't be full releases for anything but 2.7 and later. I doubt there are any fixes in 2.6 branch for scala module.

[nbauernfeind]

I'm using sbt 0.13.11, are you? Are you sure you're compiling the 2.12 branch? That can be done by either changing the version := line or by running sbt +compile to make sure you cross compile them all.