[SPARK-22781][SS] Support creating streaming dataset with ORC files

[dongjoon-hyun]

What changes were proposed in this pull request?

Like Parquet, users can use ORC with Apache Spark structured streaming. This PR adds orc() to DataStreamReader(Scala/Python) in order to support creating streaming dataset with ORC file format more easily like the other file formats. Also, this adds a test coverage for ORC data source and updates the document.

BEFORE

scala> spark.readStream.schema("a int").orc("/tmp/orc_ss").writeStream.format("console").start()
<console>:24: error: value orc is not a member of org.apache.spark.sql.streaming.DataStreamReader
       spark.readStream.schema("a int").orc("/tmp/orc_ss").writeStream.format("console").start()

AFTER

scala> spark.readStream.schema("a int").orc("/tmp/orc_ss").writeStream.format("console").start()
res0: org.apache.spark.sql.streaming.StreamingQuery = org.apache.spark.sql.execution.streaming.StreamingQueryWrapper@678b3746

scala>
-------------------------------------------
Batch: 0
-------------------------------------------
+---+
|  a|
+---+
|  1|
+---+

How was this patch tested?

Pass the newly added test cases.

[SparkQA]

Test build #84892 has finished for PR 19975 at commit 66475b7.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #84893 has finished for PR 19975 at commit 1ab78ed.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #84901 has finished for PR 19975 at commit 29094cc.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Hi, @tdas and @zsxwing .
Could you review this PR?

[dongjoon-hyun]

Also, @brkyvz . Could you review this PR?

[dongjoon-hyun]

Hi, @tdas , @zsxwing , @brkyvz .
Could you give me some advice about how to proceed this PR?

[brkyvz]

This LGTM. @zsxwing Any other comments?

[dongjoon-hyun]

Thank you so much, @brkyvz !

[zsxwing]

LGTM. Let's trigger a new build since it's 5 days old now.

retest this please.

[dongjoon-hyun]

Thank you so much, @zsxwing !

[dongjoon-hyun]

Retest this please.

[SparkQA]

Test build #85078 has finished for PR 19975 at commit 29094cc.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Retest this please

[SparkQA]

Test build #85098 has finished for PR 19975 at commit 29094cc.

• This patch fails due to an unknown error code, -9.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Retest this please

[SparkQA]

Test build #85117 has finished for PR 19975 at commit 29094cc.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Hi, @brkyvz and @zsxwing . It passed the Jenkins again finally.

[dongjoon-hyun]

Gentle ping! :)

[zsxwing]

Thanks! Merging to master!

[dongjoon-hyun]

Thank you so much, @zsxwing and @brkyvz