-
Notifications
You must be signed in to change notification settings - Fork 28.9k
[SPARK-6227][MLLIB][PYSPARK] Implement PySpark wrappers for SVD and PCA (v2) #17621
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Based on @MechCoder's work in #7963. |
|
Test build #75741 has finished for PR 17621 at commit
|
|
Test build #75742 has finished for PR 17621 at commit
|
|
Test build #75743 has finished for PR 17621 at commit
|
|
Test build #75744 has finished for PR 17621 at commit
|
|
Thanks for reviving this @MLnick :) |
dusenberrymw
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, great to see this being finished up, @MLnick. LGTM.
|
Thanks @MLnick ! |
|
Test build #76101 has finished for PR 17621 at commit
|
|
Test build #76102 has finished for PR 17621 at commit
|
|
If no further comments I'll merge this into branch-2.2 within a few days. |
|
Jenkins retest this please |
|
Test build #76377 has finished for PR 17621 at commit
|
|
LGTM |
…CA (v2) Add PCA and SVD to PySpark's wrappers for `RowMatrix` and `IndexedRowMatrix` (SVD only). Based on #7963, updated. ## How was this patch tested? New doc tests and unit tests. Ran all examples locally. Author: MechCoder <[email protected]> Author: Nick Pentreath <[email protected]> Closes #17621 from MLnick/SPARK-6227-pyspark-svd-pca. (cherry picked from commit db2fb84) Signed-off-by: Nick Pentreath <[email protected]>
|
Merged to master/branch-2.2. Thanks again for the original work @MechCoder! |
|
@MLnick Hi, I'm interesting in this PySpark wrapper for SVD. How many columns can this support? Cuz I see in the old document it can only support columns <1000. How about this wrapper? |
Add PCA and SVD to PySpark's wrappers for
RowMatrixandIndexedRowMatrix(SVD only).Based on #7963, updated.
How was this patch tested?
New doc tests and unit tests. Ran all examples locally.