Fix the treatment of LightGbm Evaluation Metric parameters in ML.NET … #3815
+31
−11
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…and make them conform to LightGbm
Ivanidzo4ka
reviewed
Jun 4, 2019
codemzs
added a commit
to codemzs/machinelearning
that referenced
this pull request
Jul 3, 2019
… ML.NET … (dotnet#3815)" This reverts commit 23332c1. # Conflicts: # test/Microsoft.ML.Tests/TrainerEstimators/TreeEstimators.cs
ghost
locked as resolved and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…and make them conform to LightGbm
Fixes #3761 plus some related issues discovered during investigation
There was a bug in LightGbm
EvaluateMetricTypewhere if a user specifiedEvaluateMetricType.Default, the metric would not get added to the options Dictionary, andLightGbmWrappedTrainingwould throw because of that.Secondly,
EvaluateMetricType.Defaultin LightGbm is supposed to be an empty string "" and LightGbm chooses the default metric according to the problem type when this is specified. This was not present in ML.NET parameter name mappings.EvaluateMetricType.Noneis supposed to be "None" in LightGbm but was "" in ML.NET parameter name mappings.[Update: REVERTED] Third, in ML.NET, the default
EvaluationMetricis set toEvaluateMetricType.Errorfor multiclass,EvaluationMetricType.LogLossfor binary, and so on. This leads to inconsistent behavior from the user's perspective: If a user specifiedEvaluationMetric = EvaluateMetricType.Default, the parameter passed to LightGbm would be the empty string "" but if they do not specifyEvaluationMetricat all, the parameter passed to LightGbm would be Error for multiclass, LogLoss for binary, and so on.This PR does the following:
EvaluationMetricin ML.NET toEvaluationMetricType.Default