[SPARK-6128][Streaming][Documentation] Updates to Spark Streaming Programming Guide

[tdas]

Updates to the documentation are as follows:

• Added information on Kafka Direct API and Kafka Python API
• Added joins to the main streaming guide
• Improved details on the fault-tolerance semantics

Generated docs located here
http://people.apache.org/~tdas/spark-1.3.0-temp-docs/streaming-programming-guide.html#fault-tolerance-semantics

More things to add:

• Configuration for Kafka receive rate
• May be add concurrentJobs

[tdas]

@JoshRosen

[JoshRosen]

"loose" -> "lose".

"zero-data" probably shouldn't be hyphenated. There's an extra space before the period at the end of the this sentence, too.

Typo: "Ssee".

[tdas]

Fixed.

[SparkQA]

Test build #28411 has finished for PR 4956 at commit 86c4c2a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #28417 has finished for PR 4956 at commit 04167a6.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #28418 has finished for PR 4956 at commit 380cf8d.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #28477 has finished for PR 4956 at commit debe484.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class Row(word: String)
  • class JavaSQLContextSingleton
  • public class JavaRow implements java.io.Serializable
  • You can also easily use machine learning algorithms provided by [MLlib](mllib-guide.html). First of all, there are streaming machine learning algorithms (e.g. (Streaming Linear Regression](mllib-linear-methods.html#streaming-linear-regression), [Streaming KMeans](file:///Users/tdas/Projects/Spark/spark/docs/_site/mllib-clustering.html#streaming-k-means), etc.) which can simultaneously learn from the streaming data as well as apply the model on the streaming data. Beyond these, for a much larger class of machine learning algorithms, you can learn a learning model offline (i.e. using historical data) and then apply the model online on streaming data. See the [MLlib](mllib-guide.html) guide for more details.

[JoshRosen]

"and then queried it using" -> drop the 'it'

[JoshRosen]

The file:// link here should be updated. Also, it looks like the link to Streaming Linear Regression starts with a opening paren rather than a square bracket, causing it to be misformatted in Markdown.

[tdas]

Thank you so much @JoshRosen . I am merging this to unblock the release!

[SparkQA]

Test build #28490 has finished for PR 4956 at commit 819408c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class Row(word: String)
  • class JavaSQLContextSingleton
  • public class JavaRow implements java.io.Serializable
  • You can also easily use machine learning algorithms provided by [MLlib](mllib-guide.html). First of all, there are streaming machine learning algorithms (e.g. (Streaming Linear Regression](mllib-linear-methods.html#streaming-linear-regression), [Streaming KMeans](mllib-clustering.html#streaming-k-means), etc.) which can simultaneously learn from the streaming data as well as apply the model on the streaming data. Beyond these, for a much larger class of machine learning algorithms, you can learn a learning model offline (i.e. using historical data) and then apply the model online on streaming data. See the [MLlib](mllib-guide.html) guide for more details.