Skip to content

use specialized axpy in RowMatrix for SVD #1378

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 19 commits into from
Closed

use specialized axpy in RowMatrix for SVD #1378

Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
e1db950
SPARK-1782: svd for sparse matrix using ARPACK
Jun 4, 2014
96d2ecb
improve eigenvalue sorting
Jun 4, 2014
fe983b0
improve scala style
Jun 4, 2014
9c80515
use non-sparse implementation when k = n
Jun 4, 2014
827411b
fix EOF new line
Jun 4, 2014
e7850ed
use aggregate and axpy
Jun 4, 2014
4c7aec3
improve comments
Jun 7, 2014
eb15100
fix binary compatibility
Jun 13, 2014
819824b
add flag for dense svd or sparse svd
Jun 18, 2014
5543cce
improve svd api
Jun 23, 2014
7148426
improve RowMatrix multiply
Jun 26, 2014
c273771
automatically determine SVD compute mode and parameters
Jul 7, 2014
62969fa
use BDV directly in symmetricEigs
mengxr Jul 9, 2014
861ec48
simplify axpy
mengxr Jul 9, 2014
a461082
make superscript show up correctly in doc
mengxr Jul 9, 2014
4c618e9
Merge pull request #1 from mengxr/vrilleup-master
vrilleup Jul 9, 2014
7312ec1
very minor comment fix
Jul 9, 2014
5255f2a
Merge remote-tracking branch 'upstream/master'
Jul 10, 2014
6fb01a3
use specialized axpy in RowMatrix
Jul 11, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,13 @@ class RowMatrix(
seqOp = (U, r) => {
val rBrz = r.toBreeze
val a = rBrz.dot(vbr.value)
brzAxpy(a, rBrz, U.asInstanceOf[BV[Double]])
rBrz match {
// use specialized axpy for better performance
case _: BDV[_] => brzAxpy(a, rBrz.asInstanceOf[BDV[Double]], U)
case _: BSV[_] => brzAxpy(a, rBrz.asInstanceOf[BSV[Double]], U)
case _ => throw new UnsupportedOperationException(
s"Do not support vector operation from type ${rBrz.getClass.getName}.")
}
U
},
combOp = (U1, U2) => U1 += U2
Expand Down