Skip to content

Conversation

@yinxusen
Copy link
Contributor

https://issues.apache.org/jira/browse/SPARK-11987

For QuantileDiscretizer, I add a QuantileDiscretizerModel in it as a nested class. The reason is a QuantileDiscretizer generates a Bucketizer mode, while the latter extends from a JavaTransformer in PySpark rather than a JavaModel, which prevents us transfering java model to python model easily. It also prevents us to call _call_java("get_splits") from outside since it is a private method.

I tried to rewrite the Bucketizer into a JavaModel in the first time, but it may hurt the current interfaces since JavaModel and JavaTransformer have different ways to set uid and different construction method args __init__(...)

@SparkQA
Copy link

SparkQA commented Nov 26, 2015

Test build #46772 has finished for PR 10007 at commit 05f3edd.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class QuantileDiscretizer(JavaEstimator, HasInputCol, HasOutputCol):\n * class QuantileDiscretizerModel(JavaModel):\n * class ChiSqSelector(JavaEstimator, HasFeaturesCol, HasOutputCol, HasLabelCol):\n * class ChiSqSelectorModel(JavaModel):\n

@SparkQA
Copy link

SparkQA commented Nov 27, 2015

Test build #46802 has finished for PR 10007 at commit 3789867.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 3, 2015

Test build #47136 has finished for PR 10007 at commit 3a33327.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class QuantileDiscretizer(JavaEstimator, HasInputCol, HasOutputCol):\n * class QuantileDiscretizerModel(JavaModel):\n * class ChiSqSelector(JavaEstimator, HasFeaturesCol, HasOutputCol, HasLabelCol):\n * class ChiSqSelectorModel(JavaModel):\n

@yinxusen
Copy link
Contributor Author

yinxusen commented Dec 8, 2015

close it for duplication

@yinxusen yinxusen closed this Dec 8, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants