export ONNX failed with Transfering a Model from PyTorch to Caffe2 and Mobile using ONNX

[kismeter]

followed with Transfering a Model from PyTorch to Caffe2 and Mobile using ONNX, when run below code, it appear AssertionError: Arrays are not almost equal to 3 decimals

import onnx
import caffe2.python.onnx.backend as onnx_caffe2_backend

# Load the ONNX ModelProto object. model is a standard Python protobuf object
model = onnx.load("super_resolution.onnx")

# prepare the caffe2 backend for executing the model this converts the ONNX model into a
# Caffe2 NetDef that can execute it. Other ONNX backends, like one for CNTK will be
# availiable soon.
prepared_backend = onnx_caffe2_backend.prepare(model)

# run the model in Caffe2

# Construct a map from input names to Tensor data.
# The graph of the model itself contains inputs for all weight parameters, after the input image.
# Since the weights are already embedded, we just need to pass the input image.
# Set the first input.
W = {model.graph.input[0].name: x.data.numpy()}

# Run the Caffe2 net:
c2_out = prepared_backend.run(W)[0]

# Verify the numerical correctness upto 3 decimal places
np.testing.assert_almost_equal(torch_out.data.cpu().numpy(), c2_out, decimal=3)

print("Exported model has been executed on Caffe2 backend, and the result looks good!")

[NoreenMekky]

I have the same issue, have you figured it out?

[liute62]

same here.

[PyojinKim]

same here.

[busixingxing]

This example https://pytorch.org/tutorials/advanced/super_resolution_with_caffe2.html
I think for the super-resolution example. The original model is trained on PyTorch 0.4.0. From my end, the program will fail on PyTorch 1.0 (not sure if 0.4.1 works), because np.testing.assert_almost_equal(torch_out.data.cpu().numpy(), c2_out, decimal=3) will check if the result match from PyTorch and Caffe2. Go back downgrade your PyTorch to 0.4.0 is my suggestion.

[liute62]

This example https://pytorch.org/tutorials/advanced/super_resolution_with_caffe2.html
I think for the super-resolution example. The original model is trained on PyTorch 0.4.0. From my end, the program will fail on PyTorch 1.0 (not sure if 0.4.1 works), because np.testing.assert_almost_equal(torch_out.data.cpu().numpy(), c2_out, decimal=3) will check if the result match from PyTorch and Caffe2. Go back downgrade your PyTorch to 0.4.0 is my suggestion.

But if go back to PyTorch 0.4.0, the torch.onnx library will be missing?

[busixingxing]

This example https://pytorch.org/tutorials/advanced/super_resolution_with_caffe2.html
I think for the super-resolution example. The original model is trained on PyTorch 0.4.0. From my end, the program will fail on PyTorch 1.0 (not sure if 0.4.1 works), because np.testing.assert_almost_equal(torch_out.data.cpu().numpy(), c2_out, decimal=3) will check if the result match from PyTorch and Caffe2. Go back downgrade your PyTorch to 0.4.0 is my suggestion.

But if go back to PyTorch 0.4.0, the torch.onnx library will be missing?

You probably need to install onnx and caffe2 by yourself. In PyTorch 1.0, caffe2 will be installed automatically if when you install PyTorch 1.0, but not for PyTorch 0.4.0.

[sakaia]

This problem still continues on PyTorch 1.1.0.
For output the tensors, the difference of torch_out.data.cpu().numpy() - c2_out is follows

[[[[ 2.3841858e-07  6.3217402e-02  1.1243662e+00 ... -8.9683622e-02
     2.6801687e-01 -7.4505806e-09]
   [ 7.9998171e-01  1.7097338e+00  1.0237219e+00 ... -7.1387494e-01
    -2.9971829e-01  1.4464737e-01]
   [ 3.9282966e-01  1.3937118e+00  9.0349954e-01 ... -1.9317374e-01
    -5.8569014e-04 -4.0951855e-02]
   ...
   [ 4.8743239e-01 -7.6235253e-01  1.5097532e+00 ... -9.6285260e-01
    -7.9658145e-01 -2.5679940e-01]
   [-7.1934119e-02  3.6978781e-02  3.7387973e-01 ... -2.0878091e+00
    -5.6183082e-01 -7.7683501e-02]
   [-8.9406967e-08 -1.6647865e-01  2.1101646e-01 ... -9.6842986e-01
     1.9150561e-01 -2.9802322e-08]]]]

torch_out.data.cpu().numpy()

[[[[ 1.3028271   1.6252544   1.6412216  ...  0.02613981  0.12420221
     0.09926877]
   [ 1.3479216   1.8284438   1.8872947  ...  0.10235227  0.16995692
     0.1357063 ]
   [ 0.9195878   1.3117      1.3384204  ...  0.14105642  0.06811136
    -0.02480316]
   ...
   [-0.01527015  0.12020171  0.2695402  ... -0.07078857 -0.16860719
    -0.20923029]
   [ 0.02814725  0.2172118   0.26773268 ... -0.22437051 -0.15085398
    -0.14345102]
   [ 0.06616494  0.10274233  0.2582131  ... -0.23129843 -0.17906183
    -0.17293906]]]]

c2_out

[[[[ 1.3028269   1.562037    0.5168555  ...  0.11582343 -0.14381465
     0.09926878]
   [ 0.5479399   0.11870988  0.8635727  ...  0.8162272   0.4696752
    -0.00894106]
   [ 0.52675813 -0.0820118   0.43492085 ...  0.33423015  0.06869705
     0.0161487 ]
   ...
   [-0.50270253  0.88255423 -1.240213   ...  0.89206403  0.6279743
     0.0475691 ]
   [ 0.10008136  0.18023302 -0.10614707 ...  1.8634385   0.41097683
    -0.06576752]
   [ 0.06616503  0.26922098  0.04719664 ...  0.7371314  -0.37056744
    -0.17293903]]]]

[shbnm21]

np.testing.assert_almost_equal(torch_out.data.cpu().numpy(), c2_out, decimal=3)

This is giving error as
AtrributError 'nonetype'object has no attribute'data'

Please help to solve the error

[jspisak]

Hey - thanks for the patience on this issue. We haven't been focusing too much on PyTorch->ONNX-> C2. A few things that are happening:

1. The current ONNX tutorial is being updated by the MSFT team - Update Super Resolution Tutorial to Use ONNXRuntime #559

2. A number of improvements are being made to the ONNX exporter in PyTorch - https://github.com/pytorch/pytorch/pulls?utf8=%E2%9C%93&q=is%3Apr+onnx+is%3Aclosed+onnx

3. We are also working in an onnx-free mobile workflow that will land later this half for folks to start looking at.

Hope this helps! (If so, please close the issue.. :) )

[shbnm21]

@jspisak
I have resolved that issue
It was due to onnx.export function.
I supposed to use torch.onnx._export function but I was using torch.onnx.export
That's why there was error

But now I am facing another issue in caffe2

I am getting an error when converting onnx model to caffe2
error is in concat operator of caffe2
here is error

Original python traceback for operator 0 in network torch-jit-export_predict in exception above (most recent call last):
RuntimeError Traceback (most recent call last)
in ()
25
26 # Run the Caffe2 net:
---> 27 c2_out = prepared_backend.run(W)[0]
28
29 # Verify the numerical correctness upto 3 decimal places

3 frames
/usr/local/lib/python3.6/dist-packages/caffe2/python/workspace.py in CallWithExceptionIntercept(func, op_id_fetcher, net_name, *args, **kwargs)
209 def CallWithExceptionIntercept(func, op_id_fetcher, net_name, *args, **kwargs):
210 try:
--> 211 return func(*args, **kwargs)
212 except Exception:
213 op_id = op_id_fetcher()

RuntimeError: [enforce fail at concat_split_op.h:289] dim == dim_j. Expect dimension = 32 got 1 at axis = 0 for input: 1. The input tensors can only have different dimensions when arg 'add_axis' = 0 and along the axis = 1 <[32, 3, 256, 256]> vs <[1, 512, 8, 8]>.
Error from operator:
input: "0" input: "1" output: "24" output: "OC2_DUMMY_0" name: "" type: "Concat" arg { name: "axis" i: 1 } device_option { device_type: 0 device_id: 0 }frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7f9cb682c441 in /usr/local/lib/python3.6/dist-packages/torch/lib/libc10.so)