[SPARK-14745] [Streaming] [WIP] CEP support in Spark Streaming #12518
Conversation
Support for Pattern match on PairDStream
|
Can one of the admins verify this patch? |
|
Hello @agsachin, This is a cool idea, and we apologize that this has been sitting here for a while. Unfortunately, add-on features like this is unlikely to be core API for DStreams/DataFrames. So this is something that is best contributed to open-source as a spark-package. |
|
@tdas I wouldn't rule it out as a possibility in the future, but I agree currently the project needs to focus more on structured streaming APIs first and then think about more complicated use cases like CEP. |
|
@rxin thanks for the clarification! |
Closes apache#11610 Closes apache#15411 Closes apache#15501 Closes apache#12613 Closes apache#12518 Closes apache#12026 Closes apache#15524 Closes apache#12693 Closes apache#12358 Closes apache#15588 Closes apache#15635 Closes apache#15678 Closes apache#14699 Closes apache#9008
|
@agsachin Hi, do you plan to create a spark-package with this? thanks |
|
@rxin HI，do you have any plan on CEP ？ |
|
Hi, is there any new plan on CEP ? |
|
When to add CEP functionality to meet complex business needs |
9 participants
What changes were proposed in this pull request?
This JIRA is about how/what can we add in Spark Streaming to support CEP out of the box.
The code in this PR needs to merge up with Structured Streaming changes in 2.0. The PR is issued for now to facilitate discussion and view by other users interested in CEP and asked on spark-user group
How was this patch tested?
Manually with KafkaDirect. WIP