[SPARK-34415][ML] Randomization in hyperparameter optimization

[PhillHenry]

What changes were proposed in this pull request?

Code in the PR generates random parameters for hyperparameter tuning. A discussion with Sean Owen can be found on the dev mailing list here:

http://apache-spark-developers-list.1001551.n3.nabble.com/Hyperparameter-Optimization-via-Randomization-td30629.html

All code is entirely my own work and I license the work to the project under the project’s open source license.

Why are the changes needed?

Randomization can be a more effective techinique than a grid search since min/max points can fall between the grid and never be found. Randomisation is not so restricted although the probability of finding minima/maxima is dependent on the number of attempts.

Alice Zheng has an accessible description on how this technique works at https://www.oreilly.com/library/view/evaluating-machine-learning/9781492048756/ch04.html

Although there are Python libraries with more sophisticated techniques, not every Spark developer is using Python.

Does this PR introduce any user-facing change?

A new class (ParamRandomBuilder.scala) and its tests have been created but there is no change to existing code. This class offers an alternative to ParamGridBuilder and can be dropped into the code wherever ParamGridBuilder appears. Indeed, it extends ParamGridBuilder and is completely compatible with its interface. It merely adds one method that provides a range over which a hyperparameter will be randomly defined.

How was this patch tested?

Tests ParamRandomBuilderSuite.scala and RandomRangesSuite.scala were added.

ParamRandomBuilderSuite is the analogue of the already existing ParamGridBuilderSuite which tests the user-facing interface.

RandomRangesSuite uses ScalaCheck to test the random ranges over which hyperparameters are distributed.

[SparkQA]

Test build #135389 has finished for PR 31535 at commit 259edfe.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[PhillHenry]

@srowen Cool. Will do.

[srowen]

Jenkins retest this please

[SparkQA]

Test build #135473 has started for PR 31535 at commit 183c2cd.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40053/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40053/

[srowen]

Jenkins retest this please

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40098/

[SparkQA]

Test build #135518 has finished for PR 31535 at commit ddfe4a9.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40098/

[srowen]

Merged to master

[dongjoon-hyun]

Hi, @PhillHenry and @srowen .
This seems to break GitHub Action linter job.
Could you check the doc generation part?

[srowen]

Oh OK let me figure that out - can probably fix forward with a patch. Er, where can I see the output? I don't see it here.

[srowen]

Ah I see it:
https://github.com/apache/spark/pull/31681/checks?check_run_id=1998307491
No such file or directory @ rb_sysopen - /__w/spark/spark/docs/../examples/src/main/python/ml/model_selection_random_hyperparameters_example.py

@PhillHenry was there an additional example file that was meant to be included in the PR? if so just open another PR and I'll add it. If necessary to restore the linter soon I can temporarily remove the reference to this example.

[dongjoon-hyun]

Thanks for the analysis, @srowen !

[PhillHenry]

@srowen Odd. The file does not seem to have been pushed (multiple IntelliJ's open on the same codebase - one for Python one for Scala?). My bad. I've created a new PR at:
#31687