SPARK-1663. Corrections for several compile errors in streaming code examples, and updates to follow API changes

[srowen]

I gave the Streaming code examples, both Scala and Java, a test run today. I turned up a number of small errors, mostly compile errors in the Java examples. There were a few typos in the Scala too.

I also took the liberty of adding things like imports, since in several cases they are not obvious. Feel free to push back on some changes.

There's one thing I haven't quite addressed in the changes. JavaPairDStream uses the Java API version of Function2 in almost all cases, as JFunction2. However it uses scala.Function2 in:

  def reduceByKeyAndWindow(reduceFunc: Function2[V, V, V], windowDuration: Duration)
  :JavaPairDStream[K, V] = {
    dstream.reduceByKeyAndWindow(reduceFunc, windowDuration)
  }

Is that a typo?

Also, in Scala, I could not get this to compile:

val windowedWordCounts = pairs.reduceByKeyAndWindow(_ + _, Seconds(30), Seconds(10))
error: missing parameter type for expanded function ((x$1, x$2) => x$1.$plus(x$2))

You can see my fix below but am I missing something?

Otherwise I can say these all worked for me!

[AmplabJenkins]

Merged build triggered.

[AmplabJenkins]

Merged build started.

[techaddict]

@srowen scala error is related to #550

[techaddict]

While using these methods we have to specify the parameter type for the reduceFunc.
https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/dstream/PairDStreamFunctions.scala#L295
https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/dstream/PairDStreamFunctions.scala#L221

[AmplabJenkins]

Merged build finished. All automated tests passed.

[AmplabJenkins]

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14566/

[pwendell]

Needs a space after the , - not a big deal I can fix on merge.

[pwendell]

@srowen these are all great - thanks! I'm going to merge it.

For the reduceByKeyAndWindow issue, I think that's just an oversight that occurred during the Java 8 re-factoring, and just needs to be changed to JFunction2.

For the compile error, this is a sort of weird corner case with method overloading in Scala. I think you'll need to specify a type signature for the reduce function.

[srowen]

Sounds good @pwendell , thanks for merging with the comma fix.

Should I open a separate issue for the JFunction2 question?