[SPARK-12780] Inconsistency returning value of ML python models' properties

[yinxusen]

https://issues.apache.org/jira/browse/SPARK-12780

[SparkQA]

Test build #49244 has finished for PR 10724 at commit 993dc9c.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yinxusen]

test it please

[SparkQA]

Test build #49313 has finished for PR 10724 at commit becc49c.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yinxusen]

retest this please

[SparkQA]

Test build #49314 has finished for PR 10724 at commit becc49c.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yinxusen]

Something wrong with Jenkins?

[jkbradley]

A PR broke the build, but it should work now.

[jkbradley]

The feature.py changes look fine to me.

@davies Could you please glance at the serialization change to confirm it's OK? Thanks!

[SparkQA]

Test build #2379 has finished for PR 10724 at commit becc49c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yinxusen]

@jkbradley @holdenk As I mentioned here #10186, we can work around the Ser/De issue with type conversion. So same solution here, I remove the Scala side code. But I still think there is a problem when ser/deing Scala Array. How about we merge this first then file another PR to fix it?

[yinxusen]

File a new JIRA https://issues.apache.org/jira/browse/SPARK-12834

[SparkQA]

Test build #49454 has finished for PR 10724 at commit f5623b6.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

@yinxusen I spoke with @davies and I think it would be best to use _call_java wherever possible for simplicity. Otherwise, developers have to remember to put list() around the JVM methods they call.

Since you have created [https://github.com//pull/10772], let's use it to create the fix. @davies said your original fix (in Scala in [https://github.com/yinxusen/spark/commit/c5a170da04f9941f4515f3263939bab9886c2c85]) seemed reasonable, so perhaps use that if your new one is not working. After that gets merged, we can return to this and the other PRs.

[yinxusen]

retest it please

[SparkQA]

Test build #50111 has finished for PR 10724 at commit c0cf68c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yinxusen]

Hi @jkbradley, this is the follow-up PR of #10772, shall we merge it?

[jkbradley]

LGTM
I'll merge this with master
Thanks!