-
Notifications
You must be signed in to change notification settings - Fork 28.9k
[SPARK-6787] [ML] add read/write to estimators under ml.feature (1) #9798
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Included some code clean-up of #6665.
|
Test build #46190 has finished for PR 9798 at commit
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are already Params. This is just weird b/c the new and old model params can get out of synch. Fine for now though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I send #9839 to do minor refactor for StandardScaler, it fixed this issue.
|
@mengxr LGTM except for the Scala style issue. My comments aren't actionable unless you want to simplify the tests using testEstimatorAndModelReadWrite |
|
|
|
test this please |
1 similar comment
|
test this please |
|
Test build #46262 has finished for PR 9798 at commit
|
Add read/write support to the following estimators under spark.ml: * CountVectorizer * IDF * MinMaxScaler * StandardScaler (a little awkward because we store some params in spark.mllib model) * StringIndexer Added some necessary method for read/write. Maybe we should add `private[ml] trait DefaultParamsReadable` and `DefaultParamsWritable` to save some boilerplate code, though we still need to override `load` for Java compatibility. jkbradley Author: Xiangrui Meng <[email protected]> Closes #9798 from mengxr/SPARK-6787. (cherry picked from commit 7e987de) Signed-off-by: Xiangrui Meng <[email protected]>
|
Merged into master and branch-1.6. |
Add read/write support to the following estimators under spark.ml:
Added some necessary method for read/write. Maybe we should add
private[ml] trait DefaultParamsReadableandDefaultParamsWritableto save some boilerplate code, though we still need to overrideloadfor Java compatibility.@jkbradley