Loading Python Exported Model into TorchSharp

[jimquittenton]

Discussed in #585

^{Originally posted by jimquittenton April 27, 2022}
Hi,
I'm new to TorchSharp and am having trouble loading a python trained ResNet18 model. I've been following this article: https://github.com/dotnet/TorchSharp/blob/main/docfx/articles/saveload.md and have exported my python model using the 'save_state_dict' function in this script: https://github.com/dotnet/TorchSharp/blob/main/src/Python/exportsd.py .

In TorchSharp I have copied the ResNet model from https://github.com/dotnet/TorchSharpExamples/blob/main/src/CSharp/Models/ResNet.cs and then call the following:

int numClasses = 3;
ResNet myModel = ResNet.ResNet18(numClasses);
myModel.to(DeviceType.CPU);
myModel.load(mPath);

The load() line throws an exception with message Mismatched module state names: the target modules does not have a submodule or buffer named 'conv1.weight'.

If I examine the state_dict from 'myModel' prior to load(), it contains entries like:

{[layers.conv2d-first.weight, {TorchSharp.Modules.Parameter}]}
{[layers.bnrm2d-first.weight, {TorchSharp.Modules.Parameter}]}
{[layers.bnrm2d-first.bias, {TorchSharp.Modules.Parameter}]}
{[layers.bnrm2d-first.running_mean, {TorchSharp.torch.Tensor}]}
{[layers.bnrm2d-first.running_var, {TorchSharp.torch.Tensor}]}
{[layers.bnrm2d-first.num_batches_tracked, {TorchSharp.torch.Tensor}]}
{[layers.blck-64-0.layers.blck-64-0-conv2d-1.weight, {TorchSharp.Modules.Parameter}]}
{[layers.blck-64-0.layers.blck-64-0-bnrm2d-1.weight, {TorchSharp.Modules.Parameter}]}
{[layers.blck-64-0.layers.blck-64-0-bnrm2d-1.bias, {TorchSharp.Modules.Parameter}]}

whereas the corresponding entries prior to saving from python are:

conv1.weight     torch.Size([64, 3, 7, 7])
bn1.weight       torch.Size([64])
bn1.bias         torch.Size([64])
bn1.running_mean         torch.Size([64])
bn1.running_var          torch.Size([64])
bn1.num_batches_tracked          torch.Size([])
layer1.0.conv1.weight    torch.Size([64, 64, 3, 3])
layer1.0.bn1.weight      torch.Size([64])
layer1.0.bn1.bias        torch.Size([64])

I tried amending the ResNet.cs code to reflect the python names, but could not get them to exactly match.

I also tried calling load() with strict=false myModel.load(mPath, false);. This seemed to get past the Mismatched names exception, but throws another exception with message Too many bytes in what should have been a 7 bit encoded Int32.

I've been struggling with this for a couple of days now so would really appreciate any help you guys could offer.

Thanks
Jim

[NiklasGustafsson]

@jimquittenton - see reply in the Discussion thread.

[NiklasGustafsson]

If you have the Python source code for the model, I'd love to take a look at it.

Niklas

[jimquittenton]

Hi Niklas, the model is literally the off the shelf PyTorch ResNet18. All I have done is change the last layer to output 3 classes:

model = models.resnet18(pretrained=True)
num_ftrs = model.fc.in_features
model.fc = nn.Linear(in_features=num_ftrs, out_features=3, bias=True)
model = model.to(device)

[NiklasGustafsson]

From torchvision?

[jimquittenton]

Sorry, yes from torchvision

[NiklasGustafsson]

Okay, thanks!

The naming of submodules can be overcome, but it will take some work to get it done. The examples code (not including the layer names) was based on:

https://github.com/kuangliu/pytorch-cifar/blob/master/models/resnet.py

and I don't know if there are any other differences between that and the torchvision version.

It would be a cool thing for someone (hint, hint...) to pick this up and fix it over in the dotnet/TorchSharpExamples repo.

[NiklasGustafsson]

@jimquittenton, I created an issue to track this in TorchSharpExamples. I will also create a new issue that generalizes the problem that you identified to all pre-loaded models in torchvision.

[NiklasGustafsson]

@jimquittenton -- just as a follow-up: I validated that the ResNet model in the examples is, indeed, simplified. There are a few things that are different, but the main one is the first Conv2d, before the residual layers. The kernel size and padding are different in the examples version. This is because the CIFAR data set is 32x32 images, which is too small to work with ResNet, which expects at least 224x224. If I change things to correspond to the torchvision version (I'll push a version that you can look at, then the weights load, which gives me some comfort...

[NiklasGustafsson]

@jimquittenton -- please take a look at PR #595 (draft PR).