StratificationColumn in CrossValidation and TrainTestSplit

[rogancarr]

CrossValidation and TrainTestSplit have a parameter called StratificationColumn that is used to preserve groupings of columns across splits (as discussed in #2487). This isn't actually stratification, so we should rename the column.

This is a forked sub-issue from #2487

Related to #1204

[Ivanidzo4ka]

Do we have any idea what should be new name?

[rogancarr]

@Ivanidzo4ka good question! In the above, I've made a suggestion for "IdColumn".

[Ivanidzo4ka]

Sorry, I guess you mention it in other issue, don't see it here.
IdColumn feels blank and also doesn't reflects purpose of it.
maybe ConsistencyColumn or RetentionColumn

[rogancarr]

How about RowGroupPreservationColumn? GroupPreservationColumn? PreservationColumn?

RowSetPreservationColumn? Super explicit, and doesn't use the word "group".

[Ivanidzo4ka]

Row Set Preservation Society. That would be good name for my second album.
GroupPreservationColumn sound best for me, but would be nice to ask other people around

[justinormont]

If I heard something was renamed to IdColumn, I would assume it was the Name column.

Is there another industry term for this? We can't be the first.

[justinormont]

Closest I see in scikit-learn is GroupShuffleSplit. Perhaps SplitGroup?

https://scikit-learn.org/stable/modules/cross_validation.html#group-shuffle-split

Another route is to rename the Group column to RankingGroup, which then frees up Stratification to move to Group (which seems to be the industry term).

[justinormont]

Speaking of renaming. @Dmitry-A was saying earlier today that Name may be better called RowID

[Ivanidzo4ka]

public TrainTestData TrainTestSplit(IDataView data, double testFraction = 0.1, string stratificationColumn = null, uint? seed = null)

public CrossValidationResult<CalibratedBinaryClassificationMetrics>[] CrossValidate( IDataView data, IEstimator<ITransformer> estimator, int numFolds = 5, string labelColumn = DefaultColumnNames.Label,string stratificationColumn = null, uint? seed = null)

@justinormont what Name are you talking about?

[justinormont]

The column purpose of Name, which allows a user to identify the row of data. It's mainly used for debugging as it's printed to the .inst.txt file. It lets you match the input data row to the output score.

I'm unsure we have brought the concept to ML.NET.

[Ivanidzo4ka]

Ah, that Name. Do we even expose it anywhere in Ml.Net? It's probably part of some commands, but I don't think we do anything with commands right now, since they all hidden

[justinormont]

I see it listed here:

machinelearning/src/Microsoft.ML.Data/Commands/DefaultColumnNames.cs

Line 12 in 0c62e30

public const string Name = "Name";

No idea if we utilize the concept though.