Early stopping support classifier and regressor fit

[mmccarty]

This change adds support for early stopping to the sklearn interface by passing arguments in the proper format.

• Classifier
• Regressor

fixes #38

[mmccarty]

It would be better to refactor certain parts of sklearn API code in xgboost to be more reusable here rather than duplicating the code. Once this settles out, I would like to clean that up.

[mmccarty]

@TomAugspurger Thanks for taking a look. If folks are good with this approach, I will proceed.

[TomAugspurger]

So to summarize the discussion from https://github.com/dask/dask-xgboost/pull/54/files/d10bb355a1d82a89233c58295c2b515d3e39baed#diff-66bc4b86e5e634f64f45b16394051674, we decided that eval_set and sample_weight_eval_set should be in-memory ndarrays, which are sent to each worker? That matches what we do with Hyperband in dask-ml.

Eventually we'll want to support sample_weight, which IIUC will be a dask array the same length as X and y.

[TomAugspurger]

Overall, things are looking good here @mmccarty. I think just linting issues now.

[mmccarty]

Great! Thank you for checking. I’ll get the linting fixed ASAP.

…

On Tue, Oct 1, 2019 at 6:23 PM Tom Augspurger ***@***.***> wrote: Overall, things are looking good here @mmccarty <https://github.com/mmccarty>. I think just linting issues now. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54?email_source=notifications&email_token=AAEY2GUZW347YUTSLIS7OITQMPEUJA5CNFSM4I2EVIZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAC6L3I#issuecomment-537257453>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEY2GQZI4QMDMT4SRZYFF3QMPEUJANCNFSM4I2EVIZA> .

[mmccarty]

@TomAugspurger CI is passing. I also want to add this to the Regressor. I'm working on that now.

[mmccarty]

@TomAugspurger @mrocklin All done here, unless there are further comments.

[TomAugspurger]

Can you check: what happens if the user does eval_set=[(x1, y2), (x2, y2)], sample_weight_eval_set=[weight1]? Do we sielently drop the (x2, y2) eval set here, since the lengths don't match?

Would the equivalent mistake raise in xgboost, or do they silently proceed?

[mmccarty]

I'll check.

[mmccarty]

@TomAugspurger I copied this code from xgboost here and then cleaned it up a bit. The original code will silently proceed. This code will error. I'm going to align it with the original code.

[mmccarty]

Actually, no it doesn't error. They behave the same. I'll push up another unit test to confirm.

[TomAugspurger]

Heh, OK.

FWIW, I think raising is the right thing to do... Maybe open an issue with XGBoost to see if they agree? Then you won't need to change anything here.

[mmccarty]

That's what I was wondering. What's the right thing to do? Three options, maybe?

1. Silently drop
2. Silently fill right
3. Raise error

[mmccarty]

@TomAugspurger I opened an issue. Good catch!

[TomAugspurger]

Alrightly, let's merge this. We can followup if xgboost decides to change their behavior.

Thanks @mmccarty!

[TomAugspurger]

Is there a time where a release would be good for you? I can do one sometime next week if that works.

[mmccarty]

Great! Thanks! Yeah, sometime next week works!