Removing ROSE class and add a shrinkage parameter to RandomOverSampler

[glemaitre]

I was checking the state of our documentation and I found that we were missing a bit of information regarding ROSE.
While looking at the article and the implementation, it seems to me that ROSE is just doing the same as the RandomOverSampler but multiply the bootstrap sample by a small random multivariate normal noise controlled that is further controlled by the shrinkage_factor parameter.

I am not a big fan to have 2 independents classes for the same processing and I would like to move the shrinkage into the RandomOverSampler as a parameter. By default, the shrinkage will be 0 such that we don't generate a smooth bootstrap to keep the original behaviour and document that this shrinkage allows generating a smooth bootstrap.

@hayesall @chkoar WDYT?

Be aware that I would like to do that before the release to not have to handle any deprecation.

[hayesall]

Shrinkage in RandomOverSampler could be cleaner. I can look in ~6 hours from now.

@andrealorenzon spoke with the authors, might have some thoughts on this too.

[glemaitre]

I think that the implementation is straightforward. I will able to make a PR tonight as well.

[glemaitre]

I made the basic changes there: https://github.com/scikit-learn-contrib/imbalanced-learn/pull/791/files#diff-3c5099e1982c8a45953caa40e38fe858a0d9a90de1253f5a44dfc4d2ffc4e64e

I need to add several tests and ensure that we raise the proper error and check the X matrix to be numerical since we need it to be for creating the smoothed bootstrap. Then, it will remain to bullet-proof the documentation.

I am still open regarding the naming of the parameters (I am not entirely sure about smoothed_bootstrap even it is quite explicit).

[chkoar]

Shrinkage in RandomOverSampler could be cleaner.

I agree. However I don't think that we need two parameters in order to enable ROSE, do we?

[glemaitre]

Reopening even if I merge the PR. It is in good condition to continue the discussion.

Indeed, we could only have the shrinkage parameter where None refer to not any smoothed_bootstrap. I choose to not do so at first because the shrinkage is indeed apply to the covariance matrix used to generate the perturbation and I was not sure that it was enough explicit. But it could be enough.

[glemaitre]

I will make a PR such that we can decide whether or not to go with a single parameter

[chkoar]

Indeed, we could only have the shrinkage parameter where None refer to not any smoothed_bootstrap.

Basically, I thought the same. Grid search was the first thing that came to my mind.
As commented here, is the shrinkage parameter unbounded? Can be negative? Do we check it? I haven't read the paper yet.

[glemaitre]

negative does not make sense here. It should be >= 0. I solved this issue in the new PR.

[andrealorenzon]

Sorry for the late comment. The only issue I see is that, in ROSE, the shrinkage parameter is class-specific, so every class can have a different shrinking factor for the Gaussian noise. At the moment, the ROSE algorithm is not the same.

[chkoar]

Sorry for the late comment. The only issue I see is that, in ROSE, the shrinkage parameter is class-specific, so every class can have a different shrinking factor for the Gaussian noise. At the moment, the ROSE algorithm is not the same.

It seems that it can be different.