[SPARK-14812][ML][MLLIB][PYTHON] Experimental, DeveloperApi annotation audit for ML

[jkbradley]

What changes were proposed in this pull request?

General decisions to follow, except where noted:

• spark.mllib, pyspark.mllib: Remove all Experimental annotations. Leave DeveloperApi annotations alone.
• spark.ml, pyspark.ml
  ** Annotate Estimator-Model pairs of classes and companion objects the same way.
  ** For all algorithms marked Experimental with Since tag <= 1.6, remove Experimental annotation.
  ** For all algorithms marked Experimental with Since tag = 2.0, leave Experimental annotation.
• DeveloperApi annotations are left alone, except where noted.
• No changes to which types are sealed.

Exceptions where I am leaving items Experimental in spark.ml, pyspark.ml, mainly because the items are new:

• Model Summary classes
• MLWriter, MLReader, MLWritable, MLReadable
• Evaluator and subclasses: There is discussion of changes around evaluating multiple metrics at once for efficiency.
• RFormula: Its behavior may need to change slightly to match R in edge cases.
• AFTSurvivalRegression
• MultilayerPerceptronClassifier

DeveloperApi changes:

• ml.tree.Node, ml.tree.Split, and subclasses should no longer be DeveloperApi

How was this patch tested?

N/A

Note to reviewers:

• spark.ml.clustering.LDA underwent significant changes (additional methods), so let me know if you want me to leave it Experimental.
• Be careful to check for cases where a class should no longer be Experimental but has an Experimental method, val, or other feature. I did not find such cases, but please verify.

[SparkQA]

Test build #62132 has finished for PR 14147 at commit f86ea5a.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

CC: @MLnick @yanboliang If you have a chance

[SparkQA]

Test build #62184 has finished for PR 14147 at commit 5bd0d38.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #62193 has finished for PR 14147 at commit 8f5fb63.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yanboliang]

@jkbradley I'd like to leave ml.clustering.LDA Experimental, since we did significant changes in this release cycle. Other than that, this looks great. Thanks!

[jkbradley]

@yanboliang Thanks for taking a look! I reverted the LDA changes for spark.ml.

[SparkQA]

Test build #62266 has finished for PR 14147 at commit b57f398.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

Merging with master and branch-2.0