-
Notifications
You must be signed in to change notification settings - Fork 28.9k
[SPARK-12711][ML] ML StopWordsRemover does not protect itself from column name duplication #10741
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Could you please add tag "[ML]" to the PR title? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo: "outputCol"
|
Also, please add a (short) PR description (in your first PR comment) since that will become part of the commit message. |
|
Test build #2377 has finished for PR 10741 at commit
|
…lumn name duplication Fixes problem and verifies fix by test suite. Also - adds optional parameter nullable (Boolean) to: SchemaUtils.appendColumn and deduplicates SchemaUtils.appendColumn functions.
|
Is everything ok with this PR? |
|
LGTM |
|
Test build #2461 has finished for PR 10741 at commit
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just copy one of the datasets from an above test. That should fix the error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I missed out that second column in dataSet was totally empty... - and that was the problem...
I do not want to make that example too complicated, because this test does not even check correctness of execution result
I'm sorry for problems
|
ok to test |
|
LGTM pending tests |
|
Test build #50573 has finished for PR 10741 at commit
|
|
Merging with master and branch-1.6 |
…lumn name duplication Fixes problem and verifies fix by test suite. Also - adds optional parameter: nullable (Boolean) to: SchemaUtils.appendColumn and deduplicates SchemaUtils.appendColumn functions. Author: Grzegorz Chilkiewicz <[email protected]> Closes #10741 from grzegorz-chilkiewicz/master. (cherry picked from commit b1835d7) Signed-off-by: Joseph K. Bradley <[email protected]>
Fixes problem and verifies fix by test suite.
Also - adds optional parameter: nullable (Boolean) to: SchemaUtils.appendColumn
and deduplicates SchemaUtils.appendColumn functions.