[SPARK-11892][ML] Model export/import for spark.ml: OneVsRest #9934

yinxusen · 2015-11-24T14:19:46Z

What changes were proposed in this pull request?

https://issues.apache.org/jira/browse/SPARK-11892

Add save/load for spark ml.OneVsRest and its model. Also add OneVsRest and OneVsRestModel in MetaAlgorithmReadWrite.

How was this patch tested?

Test with Scala unit test.

SparkQA · 2015-11-24T15:16:15Z

Test build #46604 has finished for PR 9934 at commit d24dcf8.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n * trait ClassifierTypeTrait\n

yinxusen · 2016-01-11T09:37:48Z

Updated for 2.0 cc @jkbradley

SparkQA · 2016-01-11T10:28:26Z

Test build #49131 has finished for PR 9934 at commit 4996e2f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

jkbradley · 2016-03-23T19:33:53Z

I'll take a look now

jkbradley · 2016-03-23T19:35:08Z

@yinxusen Actually, should I wait to review this until #9971 gets merged? Which should come first?

SparkQA · 2016-03-23T20:14:09Z

Test build #2671 has finished for PR 9934 at commit 4996e2f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

yinxusen · 2016-03-23T21:46:58Z

I'll update this after #9971 merged.

yinxusen · 2016-03-29T00:46:26Z

Refactoring this one now.

SparkQA · 2016-03-29T05:07:42Z

Test build #54411 has finished for PR 9934 at commit 2e31b85.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

yinxusen · 2016-03-29T05:58:45Z

@jkbradley Ready to review.

jkbradley · 2016-03-29T20:25:24Z

Github says this PR does not conflict with master, but when I rebase on master, it has conflicts. Does this happen to you?

yinxusen · 2016-03-29T21:54:12Z

Not happen on me. Maybe you can hold off first, I can check it later today because I am off my PC now, then ping you if it passes.

jkbradley · 2016-03-29T21:58:33Z

Sure, I'll go ahead and review though. Btw, can you add the "[ML]" tag to the PR title?

yinxusen · 2016-03-29T22:11:10Z

Sure

Sent from my iPhone

On Mar 29, 2016, at 14:59, jkbradley [email protected] wrote:

Sure, I'll go ahead and review though. Btw, can you add the "[ML]" tag to the PR title?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub

yinxusen · 2016-03-30T03:23:28Z

@jkbradley I merged with master, it seems good.

SparkQA · 2016-03-30T04:03:56Z

Test build #54488 has finished for PR 9934 at commit e6b16a8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

jkbradley · 2016-03-30T19:02:15Z

mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala

+      case stage: MLWritable => // good
+      case other =>
+        throw new UnsupportedOperationException("OneVsRest write will fail " +
+          s" because it contains $name which does not implement Writable." +


"Writable" --> "MLWritable"

jkbradley · 2016-03-30T19:02:20Z

mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala

-    @Since("1.4.0")  override val uid: String,
-    @Since("1.4.0") labelMetadata: Metadata,
+    @Since("1.4.0") override val uid: String,
+    @Since("1.4.0") val labelMetadata: Metadata,


This should be private[ml]. The Since tag was a mistake and can be removed since it does not show up in the public API.

jkbradley · 2016-03-30T19:02:36Z

I think that should be it.

SparkQA · 2016-03-31T01:22:55Z

Test build #54570 has finished for PR 9934 at commit 028e6fa.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

yinxusen · 2016-03-31T16:47:22Z

@jkbradley All fixed.

jkbradley · 2016-03-31T18:16:46Z

LGTM
Merging with master
Thanks @yinxusen !

yinxusen · 2016-03-31T18:18:47Z

Thanks!

jkbradley · 2016-03-31T18:20:02Z

I made a JIRA for the Python API: https://issues.apache.org/jira/browse/SPARK-14306

yinxusen · 2016-03-31T18:59:30Z

Sure I can take it.

Sent from my iPhone

On Mar 31, 2016, at 11:21, jkbradley [email protected] wrote:

I'll make a JIRA for the Python API

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub

jkbradley · 2016-03-31T19:54:27Z

Thanks!

yinxusen · 2016-03-31T20:04:49Z

@jkbradley It seems that OneVsRest is still missing for PySpark. I find the closed PR: #6403. I can help re-submitting it with the PySpark one.

jkbradley · 2016-04-01T22:15:53Z

@yinxusen Thanks! That would be great.

yinxusen added 5 commits November 21, 2015 21:20

saved and waited for SPARK-6791

595b72f

Merge branch 'master' into SPARK-11892

fdf95ee

fix test sute

a5816f0

right save model

2c82fab

fix style

d24dcf8

yinxusen added 2 commits January 11, 2016 16:46

merge with master

f2f2200

change 1.7.0 to 2.0.0

4996e2f

Merge branch 'master' into SPARK-11892

c87c992

refactoring code

2e31b85

yinxusen changed the title ~~[SPARK-11892] Model export/import for spark.ml: OneVsRest~~ [SPARK-11892][ML] Model export/import for spark.ml: OneVsRest Mar 30, 2016

Merge branch 'master' into SPARK-11892

e6b16a8

jkbradley reviewed Mar 30, 2016
View reviewed changes

fix tests

028e6fa

asfgit closed this in 8b207f3 Mar 31, 2016

[SPARK-11892][ML] Model export/import for spark.ml: OneVsRest #9934

[SPARK-11892][ML] Model export/import for spark.ml: OneVsRest #9934

Uh oh!

Conversation

yinxusen commented Nov 24, 2015

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

SparkQA commented Nov 24, 2015

Uh oh!

yinxusen commented Jan 11, 2016

Uh oh!

SparkQA commented Jan 11, 2016

Uh oh!

jkbradley commented Mar 23, 2016

Uh oh!

jkbradley commented Mar 23, 2016

Uh oh!

SparkQA commented Mar 23, 2016

Uh oh!

yinxusen commented Mar 23, 2016

Uh oh!

yinxusen commented Mar 29, 2016

Uh oh!

SparkQA commented Mar 29, 2016

Uh oh!

yinxusen commented Mar 29, 2016

Uh oh!

jkbradley commented Mar 29, 2016

Uh oh!

yinxusen commented Mar 29, 2016

Uh oh!

jkbradley commented Mar 29, 2016

Uh oh!

yinxusen commented Mar 29, 2016

Uh oh!

yinxusen commented Mar 30, 2016

Uh oh!

SparkQA commented Mar 30, 2016

Uh oh!

jkbradley Mar 30, 2016

Choose a reason for hiding this comment

Uh oh!

jkbradley Mar 30, 2016

Choose a reason for hiding this comment

Uh oh!

jkbradley commented Mar 30, 2016

Uh oh!

SparkQA commented Mar 31, 2016

Uh oh!

yinxusen commented Mar 31, 2016

Uh oh!

jkbradley commented Mar 31, 2016

Uh oh!

yinxusen commented Mar 31, 2016

Uh oh!

jkbradley commented Mar 31, 2016

Uh oh!

yinxusen commented Mar 31, 2016

Uh oh!

jkbradley commented Mar 31, 2016

Uh oh!

yinxusen commented Mar 31, 2016

Uh oh!

jkbradley commented Apr 1, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants