ipex quantization model can't be loaded after save

[rnwang04]

Hi, when I use ipex quantization with inc, I meet a problem that quantized model can't be loaded after save.
When I save, I just call quantized.save(path)
and I get a best_configure.json under my save dir tmp_dir.
Then when I want to load it, I use

from neural_compressor.utils.pytorch import load
from neural_compressor.model.model import PyTorchModel
qmodel = PyTorchModel(load(tmp_dir, model))

and I got such error:

File "/disk0/miniconda3/envs/ruonan_nano/lib/python3.7/site-packages/bigdl/nano/deps/neural_compressor/pytorch/quantized_model.py", line 52, in _load
    qmodel = PyTorchModel(load(path, model))
  File "/disk0/miniconda3/envs/ruonan_nano/lib/python3.7/site-packages/neural_compressor/utils/pytorch.py", line 165, in load
    stat_dict = torch.load(weights_file)
  File "/disk0/miniconda3/envs/ruonan_nano/lib/python3.7/site-packages/torch/serialization.py", line 699, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/disk0/miniconda3/envs/ruonan_nano/lib/python3.7/site-packages/torch/serialization.py", line 230, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/disk0/miniconda3/envs/ruonan_nano/lib/python3.7/site-packages/torch/serialization.py", line 211, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '/disk0/binbin/MSCRED/test/ckpt/ipex_q_model/best_model.pt'

which is caused by these code:

I think model is not important, the error is irrelevant with model, and you can reproduce it by any model you like.

I wonder if it's a bug of inc, or there is anything I make mistake, like if I should not use load function in neural_compressor.utils.pytorch.
Are there any special notes on the ipex quantization model, or any corresponding tutorials or instructions?

version

neural-compressor 1.13.1
torch 1.12.1

[changwangss]

hi, please use the latest version, we not only save the best_configure.json but also best_model.pt if INC IPEX is used. we recommodate the following method for loading the int8 model now.
load function. it use the best_model.pt to recover, this is a jit model, the detail code is here

from neural_compressor.utils.pytorch import load
qmodel = load(tmp_dir, model)

if you still want to use best_configure.json to load int8 model, you can refer this example.

[rnwang04]

Thanks for reply!
I wonder does this fix contains in INC2.0?
When I use INC2.0, it seems this problem still exists that there is no best_model.pt file.

[changwangss]

Thanks for reply! I wonder does this fix contains in INC2.0? When I use INC2.0, it seems this problem still exists that there is no best_model.pt file.

INC will save it automatically if you get the quantized model and call "q_model.save("./saved_results")"
here is the save code: https://github.com/intel/neural-compressor/blob/master/neural_compressor/model/torch_model.py#L778
would you like to share your usage?