-
Notifications
You must be signed in to change notification settings - Fork 28.9k
Revert [SPARK-24882][SQL] improve data source v2 API from branch 2.4 #22388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This reverts commit e754887.
|
Test build #95912 has finished for PR 22388 at commit
|
|
This seems to be the complete revert of SPARK-24882 (#22009 and the followup #22245). Am I right? |
|
yes |
|
Test build #95967 has finished for PR 22388 at commit
|
|
Test build #95968 has finished for PR 22388 at commit
|
|
Incorrect import still may exist there. |
|
MicroBatchExecution.scala and ContinuousExecution.scala look right after the revert, although it would be helpful to understand what the diff is between this and a straight |
|
Test build #95986 has finished for PR 22388 at commit
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Thanks! Merged to 2.4.
## What changes were proposed in this pull request? As discussed in the dev list, we don't want to include #22009 in Spark 2.4, as it needs data source v2 users to change the implementation intensitively, while they need to change again in next release. ## How was this patch tested? existing tests Author: Wenchen Fan <[email protected]> Closes #22388 from cloud-fan/revert.
|
Unable to merge to master. @cloud-fan Could you submit another PR to the master branch? |
|
Ur, @gatorsmile . Why do we need to revert this from |
|
As we discussed in the dev list, we only want to revert it from 2.4. I'm closing it now. |
|
Thanks for doing this, @cloud-fan! Sorry I'm late to reply, I was at Strata all last week. |
What changes were proposed in this pull request?
As discussed in the dev list, we don't want to include #22009 in Spark 2.4, as it needs data source v2 users to change the implementation intensitively, while they need to change again in next release.
How was this patch tested?
existing tests