-
Notifications
You must be signed in to change notification settings - Fork 28.9k
[SPARK-10180] [SQL] JDBC datasource are not processing EqualNullSafe filter #8743
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Please update the pull request description to describe the changes in this patch. |
|
Will this case be covered by the end-to-end docker tests? |
|
Should I better write some test codes for this separately? |
|
Can we add a test? |
|
I added a simple test for this. I wanted to add a test including |
|
test this please |
|
It looks Jenkins does not run the test for the past commits that I made as a user not added to whitelist. Would anybody run the test for this please if it looks good? |
|
test this please |
|
Test build #46425 has finished for PR 8743 at commit
|
|
Just a question. Now it looks the PR for the end-to-end docker tests is merged. Do you think it needs all the tests for all the databases (namely |
|
@zsxwing can you review this one? |
|
@HyukjinKwon +1, I think it'd better to add tests in |
|
Hm.. actually I think adding tests more with the docker ingeration tests is a bit over-tested. It looks the comparison operators I used are all already being used in |
|
Yes and you're right. We need to clearly define the rule that decides which tests should be added in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It took me several minutes to understand it. Maybe (attr IS NULL AND value IS NULL) OR (attr <> null AND value <> null AND attr = value) is easier.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, sorry, my previous expr doesn't work.
|
I'm wondering if it's okey to add |
|
@zsxwing I will anyway resolve the conflicts first |
|
Sorry I kept merging pull requests that made your life harder. Can you bring it up to date again? I promise this is the last one. |
Conflicts: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala
|
@rxin Nothing is easy and happy new year:) I just resolved conflicts. |
|
test this please |
|
Test build #2287 has finished for PR 8743 at commit
|
|
test this please |
|
Test build #2291 has finished for PR 8743 at commit
|
|
Test build #2294 has finished for PR 8743 at commit
|
|
test this please |
|
Test build #2295 has finished for PR 8743 at commit
|
|
Thanks - I've merged this. |
This PR is followed by #8391.
Previous PR fixes JDBCRDD to support null-safe equality comparison for JDBC datasource. This PR fixes the problem that it can actually return null as a result of the comparison resulting error as using the value of that comparison.