Skip to content

Change the _maxCalibrationExamples default on CalibratorUtils #5415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 30, 2020
Merged

Change the _maxCalibrationExamples default on CalibratorUtils #5415

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0182155
Change the _maxCalibrationExamples default
antoniovs1029 Sep 30, 2020
d08a3d8
Improving comments
antoniovs1029 Sep 30, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions src/Microsoft.ML.Data/Prediction/Calibrator.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -837,8 +837,9 @@ internal static object Create(IHostEnvironment env, ModelLoadContext ctx, object
[BestFriend]
internal static class CalibratorUtils
{
// maximum number of rows passed to the calibrator.
private const int _maxCalibrationExamples = 1000000;
// Maximum number of rows to process when training the Calibrator.
// If 0, we'll actually process the whole dataset.
private const int _maxCalibrationExamples = 0;

private static bool NeedCalibration(IHostEnvironment env, IChannel ch, ICalibratorTrainer calibrator,
ITrainer trainer, IPredictor predictor, RoleMappedSchema schema)
Expand Down Expand Up @@ -988,6 +989,10 @@ public static ICalibrator TrainCalibrator(IHostEnvironment env, IChannel ch, ICa
caliTrainer.ProcessTrainingExample(score, label > 0, weight);

if (maxRows > 0 && ++num >= maxRows)
// If maxRows was 0, we'll process all of the rows in the dataset
// Notice that depending on the calibrator, "processing" might mean
// randomly choosing some of the "processed" rows
// to actually train the calibrator.
break;
}
}
Expand Down