[SPARK-32092][ML][PySpark][3.0] Removed foldCol related code #29524
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
ok to test |
|
Test build #127812 has finished for PR 29524 at commit
|
Author
|
Ended up changing more than just There was a bug in #29445 where I modified the unit tests to make sure |
Author
|
cc @huaxingao |
|
Test build #127818 has finished for PR 29524 at commit
|
dongjoon-hyun
changed the title
Member
|
Also, cc @srowen |
viirya
approved these changes
Aug 24, 2020
huaxingao
approved these changes
Aug 24, 2020
huaxingao
pushed a commit
that referenced
this pull request
Aug 24, 2020
### What changes were proposed in this pull request? - Removed `foldCol` related code introduced in #29445 which is causing issues in the base branch. - Fixed `CrossValidatorModel.copy()` so that it correctly calls `.copy()` on the models instead of lists of models. ### Why are the changes needed? - `foldCol` is from 3.1 hence causing tests to fail. - `CrossValidatorModel.copy()` is supposed to shallow copy models not lists of models. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Existing tests created in #29445 ran and passed. - Updated `test_copy` to make sure `copy()` is called on models instead of lists of models. Closes #29524 from Louiszr/remove-foldcol-3.0. Authored-by: Louiszr <[email protected]> Signed-off-by: Huaxin Gao <[email protected]>
Contributor
|
Merged to 3.0. Thank you all! |
huaxingao
pushed a commit
that referenced
this pull request
Aug 28, 2020
… to copy models instead of list ### What changes were proposed in this pull request? Fixed `CrossValidatorModel.copy()` so that it correctly calls `.copy()` on the models instead of lists of models. ### Why are the changes needed? `copy()` was first changed in #29445 . The issue was found in CI of #29524 and fixed. This PR introduces the exact same change so that `CrossValidatorModel.copy()` and its related tests are aligned in branch `master` and branch `branch-3.0`. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Updated `test_copy` to make sure `copy()` is called on models instead of lists of models. Closes #29553 from Louiszr/fix-cv-copy. Authored-by: Louiszr <[email protected]> Signed-off-by: Huaxin Gao <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
foldColrelated code introduced in [SPARK-32092][ML][PySpark] Fix parameters not being copied in CrossValidatorModel.copy(), read() and write() #29445 which is causing issues in the base branch.CrossValidatorModel.copy()so that it correctly calls.copy()on the models instead of lists of models.Why are the changes needed?
foldColis from 3.1 hence causing tests to fail.CrossValidatorModel.copy()is supposed to shallow copy models not lists of models.Does this PR introduce any user-facing change?
No
How was this patch tested?
test_copyto make surecopy()is called on models instead of lists of models.