Skip to content

Conversation

@shahidki31
Copy link
Contributor

@shahidki31 shahidki31 commented Oct 6, 2018

… with intercept with elasticnet regularization

What changes were proposed in this pull request?

In the test, "multinomial logistic regression with intercept with elasticnet regularization" in the "LogisticRegressionSuite", taking around 1 minute to train 2 logistic regression model.
However after analyzing the training cost over iteration, we can reduce the computation time by 50%.
Training cost vs iteration for model 1
image

So, model1 is converging after iteration 200.

Training cost vs iteration for model 2:
image
After around 50 iteration, model2 is converging.
So, if we give maximum iteration for model1 and model2 as 220 and 90 respectively, we can reduce the computation time by half.

How was this patch tested?

Computation time in local setup :
Before change:
~54 sec
After change:
~35 sec

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

Please review http://spark.apache.org/contributing.html before opening a pull request.

@AmplabJenkins
Copy link

Can one of the admins verify this patch?

… with intercept with elasticnet regularization 56 seconds
@shahidki31
Copy link
Contributor Author

shahidki31 commented Oct 7, 2018

cc @srowen Kindly review. I have done the same approach in the PR, #22659

@srowen
Copy link
Member

srowen commented Oct 7, 2018

This kind of thing looks OK, but please make one PR. There's no point in opening lots of JIRAs and PRs for the same change in N places.

@shahidki31
Copy link
Contributor Author

Thanks for the suggestion. I will close this and amend in the other PR.

@shahidki31 shahidki31 closed this Oct 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants