[SPARK-16114] [SQL] updated structured streaming guide #14183
Conversation
jjthomas
changed the title
| ).toDF("word", "timestamp") | ||
|
|
||
| // Group the data by window and word and compute the count of each group | ||
| val windowedCounts = words.groupBy( |
There was a problem hiding this comment.
I took a look at the built doc again and imagined what it would look like. This would look very verbose. I think since the nearest example in the doc (Basic Operations - Selection, Projection, Aggregation) uses device data and already has all the boilerplate code to define DeviceData class, etc., lets not change the code snippet to the exact one in the example.
Can you revert all the code snippet changes, and just do one change for the Scala snippet.
- Change df.col("..") to use $"..."
- Add import spark.implicits._
|
ok to test |
| .avg("signal") | ||
| // Group the data by window and word and compute the count of each group | ||
| val windowedCounts = words.groupBy( | ||
| window($"timestamp", "10 minutes", "5 minutes"), $"word" |
There was a problem hiding this comment.
put word on next line, to be consistent with other examples.
|
LGTM. |
|
Merging to master and 2.0. Tests dont matter here. |
## What changes were proposed in this pull request? Updated structured streaming programming guide with new windowed example. ## How was this patch tested? Docs Author: James Thomas <[email protected]> Closes #14183 from jjthomas/ss_docs_update. (cherry picked from commit 51a6706) Signed-off-by: Tathagata Das <[email protected]>
|
Test build #62276 has finished for PR 14183 at commit
|
|
Test build #62277 has finished for PR 14183 at commit
|
3 participants
What changes were proposed in this pull request?
Updated structured streaming programming guide with new windowed example.
How was this patch tested?
Docs