LightGBM parameter changes to match Python implementation results

[torronen]

Suggested changes for LightGBM results through ML.NET similar as through Python:

• keep LightGBM default seed if seed has not been set
• add mapping from NumberOfIterations to num_iterations
• add NumberOfIterations to parameters array for LightGBM
• change sigmoid default value to match LightGBM
• Default Evaluation Metric to None per LightGBM default

Project that can be used for comparison between LightGBM in Python and Microsoft.ML.LightGBM and also compare ModelBuilder with python-FLAML: https://github.com/torronen/lightgbm-comparison

Rationale: microsoft/FLAML#409 (comment)
Reasons for changes explained in the issues:

• fixes Q: Where is num_iterations set for LightGBM binary? #6063
• fixes LightGBM should use default seed if not set explicitly #6062
• fixes Default value of sigmoid parameter for LightGBM #654

I suggest the results should be equal through Python and ML.NET so that developers can discuss and share best practices about hyperparameters. Also, it enables to use tuning from Python.

Sigmoid value change has been propose before but was not implemented.
It may need more consideration: #667

PR is for comments and discussions for now. Results are not yet equal through ML.Net and Python.

[dnfadmin]

All CLA requirements met.

[torronen]

These names are valid aliases. Defaults are not yet considered, but at least metric should be "" per default, not logloss, but it might not matter too much.

Missing:

• min_child_samples
• learning_rate
  They are set here:

  machinelearning/src/Microsoft.ML.LightGbm/LightGbmTrainerBase.cs

  Lines 431 to 433 in 8eac93a

  	GbmOptions["learning_rate"] = learningRate;
  	GbmOptions["num_leaves"] = numberOfLeaves;
  	GbmOptions["min_data_per_leaf"] = minimumExampleCountPerLeaf;

[torronen]

Is there any reason to use aliases? If not, I suggest we update

• main_split_gain=main_split_to_gain
• min_sum_hessian_in_leaf=min_child_weight
• bagging_freq = subsample_freq
• bagging_fraction = subsample
• lambda_l2 =reg_lambda
• lambda_l1 =reg_alpha
• boosting =boosting_type
• verbosity =verbose
• unbalanced_sets = is_unbalance
• min_data_in_leaf = min_data_per_leaf

TODO: Check names are valid for 2.3.1, above is from current documentation.
2.3.1 source: https://github.com/microsoft/LightGBM/blob/v2.3.1/src/io/config_auto.cpp

[torronen]

Iterations actually seem to be ran inside .NET, so it does not need to be passed anywhere.

machinelearning/src/Microsoft.ML.LightGbm/WrappedLightGbmTraining.cs

Line 19 in 8eac93a

Dictionary<string, object> parameters, Dataset dtrain, Dataset dvalid = null, int numIteration = 100,

I will close this PR as it is better not to update the defaults if it does not provide any better performance. Changing default may be a breaking change for some developers.

However, for some reason Python seems to provide better speed and accuracy for LightGBM (and therefore, for many applications) at least on a few datasets I've tried.