-
Notifications
You must be signed in to change notification settings - Fork 28.9k
[SPARK-9690][ML][PYTHON] pyspark CrossValidator random seed #10268
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Test build #47591 has finished for PR 10268 at commit
|
|
Test build #2210 has finished for PR 10268 at commit
|
python/pyspark/ml/tuning.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add \ at the end of line, otherwise it can not generate python doc correctly.
|
LGTM except minor issues. |
|
@yanboliang Thanks! With the fix, it worked when I generated the docs locally. |
|
Test build #47766 has finished for PR 10268 at commit
|
|
Test build #2219 has finished for PR 10268 at commit
|
python/pyspark/ml/tuning.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We set default value of seed to this.getClass.getName.hashCode.toLong at Scala side. I think we should keep consistent with that, otherwise users may get different result when training with the same dataset.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is just an example though. But I agree we could remove the seed now to simplify the example. I'll do that.
|
@jkbradley I think it's the last issue that we should resolve, thanks! |
|
Test build #47844 has finished for PR 10268 at commit
|
|
Merged with master |
Extend CrossValidator with HasSeed in PySpark.
This PR replaces [https://github.com//pull/7997]
CC: @yanboliang @thunterdb @mmenestret Would one of you mind taking a look? Thanks!