Skip to content

[SPARK-21087] [ML] [FOLLOWUP] Sync SharedParamsCodeGen and sharedParams. #19958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

[SPARK-21087] [ML] [FOLLOWUP] Sync SharedParamsCodeGen and sharedParams. #19958

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,10 @@ private[shared] object SharedParamsCodeGen {
ParamDesc[String]("solver", "the solver algorithm for optimization", finalFields = false),
ParamDesc[Int]("aggregationDepth", "suggested depth for treeAggregate (>= 2)", Some("2"),
isValid = "ParamValidators.gtEq(2)", isExpertParam = true),
ParamDesc[Boolean]("collectSubModels", "If set to false, then only the single best " +
"sub-model will be available after fitting. If set to true, then all sub-models will be " +
"available. Warning: For large models, collecting all sub-models can cause OOMs on the " +
"Spark driver.",
ParamDesc[Boolean]("collectSubModels", "whether to collect a list of sub-models trained " +
"during tuning. If set to false, then only the single best sub-model will be available " +
"after fitting. If set to true, then all sub-models will be available. Warning: For " +
"large models, collecting all sub-models can cause OOMs on the Spark driver",
Some("false"), isExpertParam = true)
)

Expand Down
10 changes: 6 additions & 4 deletions mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -470,15 +470,17 @@ trait HasAggregationDepth extends Params {
}

/**
* Trait for shared param collectSubModels (default: false).
* Trait for shared param collectSubModels (default: false). This trait may be changed or
* removed between minor versions.
*/
private[ml] trait HasCollectSubModels extends Params {
@DeveloperApi
trait HasCollectSubModels extends Params {

/**
* Param for whether to collect a list of sub-models trained during tuning.
* Param for whether to collect a list of sub-models trained during tuning. If set to false, then only the single best sub-model will be available after fitting. If set to true, then all sub-models will be available. Warning: For large models, collecting all sub-models can cause OOMs on the Spark driver.
* @group expertParam
*/
final val collectSubModels: BooleanParam = new BooleanParam(this, "collectSubModels", "whether to collect a list of sub-models trained during tuning")
final val collectSubModels: BooleanParam = new BooleanParam(this, "collectSubModels", "whether to collect a list of sub-models trained during tuning. If set to false, then only the single best sub-model will be available after fitting. If set to true, then all sub-models will be available. Warning: For large models, collecting all sub-models can cause OOMs on the Spark driver")

setDefault(collectSubModels, false)

Expand Down