Skip to content

Conversation

@MLnick
Copy link
Contributor

@MLnick MLnick commented Apr 12, 2017

Add PCA and SVD to PySpark's wrappers for RowMatrix and IndexedRowMatrix (SVD only).

Based on #7963, updated.

How was this patch tested?

New doc tests and unit tests. Ran all examples locally.

@MLnick
Copy link
Contributor Author

MLnick commented Apr 12, 2017

Based on @MechCoder's work in #7963.

cc @holdenk @dusenberrymw

@SparkQA
Copy link

SparkQA commented Apr 12, 2017

Test build #75741 has finished for PR 17621 at commit 3ba2411.

  • This patch fails Python style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 12, 2017

Test build #75742 has finished for PR 17621 at commit 3ea88e2.

  • This patch fails Python style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 12, 2017

Test build #75743 has finished for PR 17621 at commit 0118f2c.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 12, 2017

Test build #75744 has finished for PR 17621 at commit 56205d2.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@holdenk
Copy link
Contributor

holdenk commented Apr 12, 2017

Thanks for reviving this @MLnick :)

Copy link
Contributor

@dusenberrymw dusenberrymw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, great to see this being finished up, @MLnick. LGTM.

@MechCoder
Copy link
Contributor

Thanks @MLnick !

@SparkQA
Copy link

SparkQA commented Apr 24, 2017

Test build #76101 has finished for PR 17621 at commit 07808fc.

  • This patch fails Python style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 24, 2017

Test build #76102 has finished for PR 17621 at commit 94006a4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MLnick
Copy link
Contributor Author

MLnick commented Apr 24, 2017

If no further comments I'll merge this into branch-2.2 within a few days.

@MLnick
Copy link
Contributor Author

MLnick commented May 2, 2017

Jenkins retest this please

@SparkQA
Copy link

SparkQA commented May 2, 2017

Test build #76377 has finished for PR 17621 at commit 94006a4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@holdenk
Copy link
Contributor

holdenk commented May 2, 2017

LGTM

asfgit pushed a commit that referenced this pull request May 3, 2017
…CA (v2)

Add PCA and SVD to PySpark's wrappers for `RowMatrix` and `IndexedRowMatrix` (SVD only).

Based on #7963, updated.

## How was this patch tested?

New doc tests and unit tests. Ran all examples locally.

Author: MechCoder <[email protected]>
Author: Nick Pentreath <[email protected]>

Closes #17621 from MLnick/SPARK-6227-pyspark-svd-pca.

(cherry picked from commit db2fb84)
Signed-off-by: Nick Pentreath <[email protected]>
@MLnick
Copy link
Contributor Author

MLnick commented May 3, 2017

Merged to master/branch-2.2. Thanks again for the original work @MechCoder!

@asfgit asfgit closed this in db2fb84 May 3, 2017
@SixAlien3
Copy link

@MLnick Hi, I'm interesting in this PySpark wrapper for SVD. How many columns can this support? Cuz I see in the old document it can only support columns <1000. How about this wrapper?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants