Skip to content

Commit 230bbea

Browse files
JeremyNixonmengxr
authored andcommitted
[SPARK-10759][ML] update cross validator with include_example
This pull request uses {%include_example%} to add an example for the python cross validator to ml-guide. Author: JeremyNixon <[email protected]> Closes #11240 from JeremyNixon/pipeline_include_example.
1 parent 8d29001 commit 230bbea

File tree

2 files changed

+9
-1
lines changed

2 files changed

+9
-1
lines changed

docs/ml-guide.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -283,6 +283,11 @@ However, it is also a well-established method for choosing parameters which is m
283283
{% include_example java/org/apache/spark/examples/ml/JavaModelSelectionViaCrossValidationExample.java %}
284284
</div>
285285

286+
<div data-lang="python">
287+
288+
{% include_example python/ml/cross_validator.py %}
289+
</div>
290+
286291
</div>
287292

288293
## Example: model selection via train validation split

examples/src/main/python/ml/cross_validator.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,14 @@
1818
from __future__ import print_function
1919

2020
from pyspark import SparkContext
21+
# $example on$
2122
from pyspark.ml import Pipeline
2223
from pyspark.ml.classification import LogisticRegression
2324
from pyspark.ml.evaluation import BinaryClassificationEvaluator
2425
from pyspark.ml.feature import HashingTF, Tokenizer
2526
from pyspark.ml.tuning import CrossValidator, ParamGridBuilder
2627
from pyspark.sql import Row, SQLContext
28+
# $example off$
2729

2830
"""
2931
A simple example demonstrating model selection using CrossValidator.
@@ -36,7 +38,7 @@
3638
if __name__ == "__main__":
3739
sc = SparkContext(appName="CrossValidatorExample")
3840
sqlContext = SQLContext(sc)
39-
41+
# $example on$
4042
# Prepare training documents, which are labeled.
4143
LabeledDocument = Row("id", "text", "label")
4244
training = sc.parallelize([(0, "a b c d e spark", 1.0),
@@ -92,5 +94,6 @@
9294
selected = prediction.select("id", "text", "probability", "prediction")
9395
for row in selected.collect():
9496
print(row)
97+
# $example off$
9598

9699
sc.stop()

0 commit comments

Comments
 (0)