Skip to content

Added LearningRateScheduler functionality for Image Classification #4340

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 14 commits into from
Oct 23, 2019
Merged

Added LearningRateScheduler functionality for Image Classification #4340

merged 14 commits into from
Oct 23, 2019

Conversation

@harshithapv
Copy link
Contributor

@harshithapv harshithapv commented Oct 16, 2019 •
edited
Loading

  1. Added LearningRateScheduler abstract class which will be implemented for various learning rate scheduling algorithms.
  2. Has the lastest Tensorflow nuget which allows Gradient Descent Optimizer to use learning rate input as tensor

@codecov
Copy link

codecov bot commented Oct 16, 2019 •
edited
Loading

Codecov Report

❗ No coverage uploaded for pull request base (master@0c7c28d). Click here to learn what that means.
The diff coverage is 76.81%.

@@            Coverage Diff            @@
##             master    #4340   +/-   ##
=========================================
  Coverage          ?   74.66%           
=========================================
  Files             ?      883           
  Lines             ?   155117           
  Branches          ?    16931           
=========================================
  Hits              ?   115824           
  Misses            ?    34549           
  Partials          ?     4744
Flag Coverage Δ
#Debug 74.66% <76.81%> (?)
#production 70.24% <61.01%> (?)
#test 89.56% <95.09%> (?)
Impacted Files Coverage Δ
src/Microsoft.ML.Dnn/DnnCatalog.cs 100% <100%> (ø)
src/Microsoft.ML.Dnn/LearningRateScheduler.cs 30.5% <30.5%> (ø)
...c/Microsoft.ML.Dnn/ImageClassificationTransform.cs 92.55% <91.37%> (ø)
...cenariosWithDirectInstantiation/TensorflowTests.cs 90.88% <95.09%> (ø)

codemzs
codemzs reviewed Oct 16, 2019
View reviewed changes
codemzs
codemzs reviewed Oct 16, 2019
View reviewed changes
codemzs
codemzs reviewed Oct 16, 2019
View reviewed changes
codemzs
codemzs reviewed Oct 16, 2019
View reviewed changes
codemzs
codemzs reviewed Oct 16, 2019
View reviewed changes
codemzs
codemzs reviewed Oct 16, 2019
View reviewed changes
codemzs
codemzs reviewed Oct 16, 2019
View reviewed changes
Harshitha Parnandi Venkata added 3 commits October 17, 2019 21:24
Installed Tensoflow.Net version 0.11.7 for GradientDescentOptimizer t…
111916c 
…o take learning rate as a tensor. Addressed Zeeshan's comments. Added linear scale rule LR decay method for learning rate scheduling.
Merge branch 'master' of https://github.com/dotnet/machinelearning in…
93e01db 
…to LRSchedulerCodeOnly
synced with master and editted a few comments.
d5c9a66
Oceania2018
Oceania2018 reviewed Oct 18, 2019
View reviewed changes
Harshitha Parnandi Venkata added 3 commits October 21, 2019 16:02
1. Updated TensorFlow .Net Nuget to 0.11.8.1 which fixes all the issu…
522c951 
…es with GrandientDescentOptimizer for Tensor input. 2. Added Exponential decay and Linear Scaling Decay for learning rate scheduling. Removed BasicLR class. 3. Added a sample for testing linear scaling rule and LR decay for Cifar dataset with resnet_v2_101. 4. Added a unit test to test Exponential decay.
Merge branch 'master' of https://github.com/dotnet/machinelearning in…
c861f71 
…to LRSchedulerCodeOnly
Fixed a bug that occurs while loading in-memory images
d04dbe0
@codemzs codemzs marked this pull request as ready for review October 22, 2019 17:08
@codemzs codemzs requested a review from a team as a code owner October 22, 2019 17:08
eerhardt
eerhardt reviewed Oct 22, 2019
View reviewed changes
eerhardt
eerhardt reviewed Oct 22, 2019
View reviewed changes
eerhardt
eerhardt reviewed Oct 22, 2019
View reviewed changes
Harshitha Parnandi Venkata added 2 commits October 22, 2019 15:41
eerhardt
eerhardt reviewed Oct 22, 2019
View reviewed changes
eerhardt
eerhardt reviewed Oct 22, 2019
View reviewed changes
eerhardt
eerhardt reviewed Oct 22, 2019
View reviewed changes
eerhardt
eerhardt reviewed Oct 22, 2019
View reviewed changes
eerhardt
eerhardt reviewed Oct 22, 2019
View reviewed changes
@codemzs
Copy link
Member

codemzs commented Oct 23, 2019 •
edited by harshithapv
Loading

@harshithapv Can you please update the title and description of the PR? #Resolved

@harshithapv harshithapv changed the title Added LearningRateScheduler interface and GradientDescentOptimizerTensor class Added LearningRateScheduler functionality for Image Classification Oct 23, 2019
codemzs
codemzs reviewed Oct 23, 2019
View reviewed changes
codemzs
codemzs reviewed Oct 23, 2019
View reviewed changes
src/Microsoft.ML.Dnn/ImageClassificationTransform.cs
scoreColumnName = ctx.Reader.ReadString();
predictedColumnName = ctx.Reader.ReadString();
learningRate = ctx.Reader.ReadFloat();
useLearningRateScheduling = ctx.Reader.ReadBoolByte();
Copy link
Member

@codemzs codemzs Oct 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

useLearningRateScheduling [](start = 12, length = 25)

why are we serializing this? It is only needed at training and not inferencing, right?

Copy link
Contributor Author

@harshithapv harshithapv Oct 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not be serializing this. This was discussed with Zeeshan and will be addressed by him in his PR.

In reply to: 337825750 [](ancestors = 337825750)

codemzs
codemzs reviewed Oct 23, 2019
View reviewed changes
codemzs
codemzs reviewed Oct 23, 2019
View reviewed changes
Added LearningRateSchedulerItem struct to represent epoch-scaling fac…
a62fb34 
…tor required for Linear scale rule and read them as IReadOnlyList. Addressed Zeeshan's comments.
codemzs
codemzs approved these changes Oct 23, 2019
View reviewed changes
Copy link
Member

@codemzs codemzs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@codemzs codemzs merged commit 6d192b6 into dotnet:master Oct 23, 2019
frank-dong-ms-zz pushed a commit to frank-dong-ms-zz/machinelearning that referenced this pull request Nov 4, 2019
@harshithapv @frank-dong-ms
Added LearningRateScheduler functionality for Image Classification (d…
6e6ee46 
…otnet#4340)

* Code with LearningRateScheduler class and GradientDescentOptimizer Class for allowing learning rate as Tensor.

* Installed Tensoflow.Net version 0.11.7 for GradientDescentOptimizer to take learning rate as a tensor. Addressed Zeeshan's comments. Added linear scale rule LR decay method for learning rate scheduling.

* synced with master and editted a few comments.

* 1. Updated TensorFlow .Net Nuget to 0.11.8.1 which fixes all the issues with GrandientDescentOptimizer for Tensor input. 2. Added Exponential decay and Linear Scaling Decay for learning rate scheduling. Removed BasicLR class. 3. Added a sample for testing linear scaling rule and LR decay for Cifar dataset with resnet_v2_101. 4. Added a unit test to test Exponential decay.

* Fixed a bug that occurs while loading in-memory images

* Changed LearningScheduler interface to an abstract class as discussed with Eric. Added more comments for the learning rate functions.

* Reverted LearningRateSchedulingCifarResnetTransferLearning.cs

* Fixed unit test. Addressed Eric's comments

* Added an internal constructor in LearningRateScheduler class.

* Added LearningRateSchedulerItem struct to represent epoch-scaling factor required for Linear scale rule and read them as IReadOnlyList. Addressed Zeeshan's comments.
@ghost ghost locked as resolved and limited conversation to collaborators Mar 20, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@eerhardt eerhardt eerhardt left review comments

@codemzs codemzs codemzs approved these changes

+1 more reviewer

@Oceania2018 Oceania2018 Oceania2018 left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@harshithapv @codemzs @Oceania2018 @eerhardt