Skip to content

Commit f7864d0

Browse files
committed
update auto logic for rowMatrix svd
1 parent 23860e4 commit f7864d0

File tree

1 file changed

+5
-1
lines changed
  • mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed

1 file changed

+5
-1
lines changed

mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -220,8 +220,12 @@ class RowMatrix(
220220

221221
val computeMode = mode match {
222222
case "auto" =>
223+
if(k > 5000) {
224+
logWarning(s"computing svd with k=$k and n=$n, please check necessity")
225+
}
226+
223227
// TODO: The conditions below are not fully tested.
224-
if (n < 100 || k > n / 2) {
228+
if (n < 100 || (k > n / 2 && n <= 15000)) {
225229
// If n is small or k is large compared with n, we better compute the Gramian matrix first
226230
// and then compute its eigenvalues locally, instead of making multiple passes.
227231
if (k < n / 3) {

0 commit comments

Comments
 (0)