Update PyTorch examples quantization API #192

changwangss · 2022-11-30T09:06:27Z

Signed-off-by: changwa1 [email protected]

Type of Change

API changed

Description

enable examples for recommendation, speech recognition

Expected Behavior & Potential Risk

the expected behavior that triggered by this PR

How has this PR been tested?

how to reproduce the test (including hardware information)

Dependency Change?

any library dependency introduced or removed

Signed-off-by: changwa1 <[email protected]>

chensuyue · 2022-11-30T13:24:04Z

examples/.config/model_params_pytorch.json

-      "batch_size": 16384,
-      "new_benchmark": false
+      "batch_size": 16384
    },


require new parameter: “main_script”: “xxx.py”,

Because test code need to replace some config class, so need to know the name scripts.

chensuyue · 2022-11-30T15:07:00Z

example readme need to update.
benchmark part need update into Config Class as well.
run_benchmark.sh input parameter update.
json param missing main_script.
Details pls check INC newAPI examples.pptx

) Signed-off-by: Cheng, Penghui <[email protected]>

Signed-off-by: Lv, Liang1 <[email protected]>

Signed-off-by: changwa1 <[email protected]>

* Fixed UT error for bf16 op list for QAT mode Signed-off-by: Cheng, Penghui <[email protected]>

Signed-off-by: Lv, Liang1 <[email protected]>

Signed-off-by: intel-zhangyi <[email protected]>

Signed-off-by: Wang, Chang1 <[email protected]>

Signed-off-by: chensuyue <[email protected]>

Signed-off-by: Xinyu Ye <[email protected]>

…166) Signed-off-by: Xinyu Ye <[email protected]>

Signed-off-by: changwa1 <[email protected]>

Signed-off-by: Lv, Liang1 <[email protected]> Signed-off-by: Lv, Liang1 <[email protected]>

* Fix centernet_hg104 tuning issue Signed-off-by: sys-lpot-val <[email protected]> * Fix TextRNN tuning issue Signed-off-by: sys-lpot-val <[email protected]> Signed-off-by: sys-lpot-val <[email protected]> Signed-off-by: Lv, Liang1 <[email protected]> Co-authored-by: sys-lpot-val <[email protected]>

…ers (#214) * Create intel_extension_for_transformers.yaml * change default strategy to dynamic according to huggingface sync * change default strategy to dynamic according to HF sync * enable intel extension for transformers * Create change_trainer_to_nlptrainer.py * add use_inc for not using default optimum for HF code * add use_inc * update optimum quant static dynamic separation * Update interface.py * Update interface.py * Update autoinc_harness.py * Update README.md * add change_trainer_to_nlptrainer to outside_harness * add PythonLauncher to pass spelling check CI Signed-off-by: Yue, Wenjiao <[email protected]> Signed-off-by: Yue, Wenjiao <[email protected]> Co-authored-by: Yue, Wenjiao <[email protected]>

* update launcher to fit multi-item input * Update __main__.py

…reliminary) (#217) * Create tf_inc_static_quant.yaml * Create inc.py * Delete tf_inc_static_quant.yaml * Update interface.py * Update inc.py

Signed-off-by: Sun, Xuehao <[email protected]>

Signed-off-by: zehao-intel <[email protected]>

Signed-off-by: chensuyue <[email protected]>

Signed-off-by: yiliu30 <[email protected]> Co-authored-by: lvliang-intel <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: xinhe <[email protected]> Co-authored-by: Ray <[email protected]>

Signed-off-by: Cheng, Penghui <[email protected]>

Signed-off-by: Lv, Liang1 <[email protected]> Signed-off-by: chensuyue <[email protected]> Co-authored-by: Lv, Liang1 <[email protected]> Co-authored-by: chensuyue <[email protected]>

Signed-off-by: Xin He <[email protected]> Co-authored-by: yiliu30 <[email protected]>

Signed-off-by: changwa1 <[email protected]>

Signed-off-by: chensuyue <[email protected]>

* enable auto bench in launcher * debug * debug

* add pruning v2 Signed-off-by: wenhuach21 <[email protected]> * DATASETS->Datasets Signed-off-by: wenhuach21 <[email protected]> * pruner README v2 Signed-off-by: Lu, Yintong <[email protected]> * prune README v2 Signed-off-by: Lu, Yintong <[email protected]> * prune README v2 Signed-off-by: Lu, Yintong <[email protected]> * prune README v2 Signed-off-by: Lu, Yintong <[email protected]> * prune README v2 Signed-off-by: Lu, Yintong <[email protected]> * recover code in experimental Signed-off-by: wenhuach21 <[email protected]> * recover config Signed-off-by: wenhuach21 <[email protected]> Signed-off-by: wenhuach21 <[email protected]> Signed-off-by: Lu, Yintong <[email protected]> Co-authored-by: Lu, Yintong <[email protected]>

…arch Toolkit (#197) Signed-off-by: Maciej Szankin <[email protected]> Co-authored-by: Nittur Sridhar, Sharath <[email protected]> Co-authored-by: Xinyu Ye <[email protected]>

Signed-off-by: mengniwa <[email protected]>

Signed-off-by: Lv, Liang1 <[email protected]> Co-authored-by: chen, suyue <[email protected]>

* update qa example Signed-off-by: Zhang, Weiwei1 <[email protected]> * pruning doc modify Signed-off-by: Lu, Yintong <[email protected]> * pruning doc modify Signed-off-by: Lu, Yintong <[email protected]> * pruning doc modify Signed-off-by: Lu, Yintong <[email protected]> * pruning doc modify Signed-off-by: Lu, Yintong <[email protected]> * pruning doc modify Signed-off-by: Lu, Yintong <[email protected]> Signed-off-by: Zhang, Weiwei1 <[email protected]> Signed-off-by: Lu, Yintong <[email protected]> Co-authored-by: Lu, Yintong <[email protected]>

* Fixed calibration sampling size error * Update training fit API docstring * Fixed IPEX examples error Signed-off-by: Cheng, Penghui <[email protected]>

Signed-off-by: Cheng, Penghui <[email protected]>

Signed-off-by: zehao-intel <[email protected]>

* Add recipe for TRT EP Signed-off-by: Mengni Wang <[email protected]> * remove codes Signed-off-by: Mengni Wang <[email protected]> Signed-off-by: Mengni Wang <[email protected]>

Signed-off-by: zehao-intel <[email protected]>

Signed-off-by: changwa1 <[email protected]>

* add subfunc for dense loading * fuse load dense & tileprod * load idx from mem * use subfunc_level * dump jit binary for each cores too * adopt unified bsr reordering to amx kernel * vnni kernel padding outside * chore * chore: make gcc happy * fix extra inst * smaller local dense offset & load dense first * use static member variable as dump index * keep param1 untouched * try to keep reg the same * cpplint

* Added PatchedLinearBase * Fixed PatchedLinear forward_qdq * Changed quant strategy - scale to fix ci * Renamed QuantStrategy to QuantWrapper * Removed instance member from QuantWrapper * [SW-224403] Added ticket and throwing error when using row_parallel_linear_allreduce_quantization * Changed QuantWrapper to a simple method that stores scale * [SW-224538] Added ticket to TODO comment for init_linear * Pushed requires_grad to the tensor creation * Update neural_compressor/torch/algorithms/fp8_quant/_quant_common/helper_modules.py * Update neural_compressor/torch/algorithms/fp8_quant/_quant_common/helper_modules.py * Moved copy_scale functions inside PatchedLinearBase * Update helper_modules.py * Update helper_modules.py * Update neural_compressor/torch/algorithms/fp8_quant/_quant_common/helper_modules.py * Update helper_modules.py * Update helper_modules.py * Update helper_modules.py copy scale * Update neural_compressor/torch/algorithms/fp8_quant/_quant_common/helper_modules.py

update quantization api

ba48deb

Signed-off-by: changwa1 <[email protected]>

changwangss changed the title ~~update quantization api~~ Update PyTorch exmpale quantization API Nov 30, 2022

changwangss changed the title ~~Update PyTorch exmpale quantization API~~ Update PyTorch examples quantization API Nov 30, 2022

changwangss requested a review from PenghuiCheng November 30, 2022 09:24

PenghuiCheng requested review from chensuyue and xin3he November 30, 2022 09:35

PenghuiCheng approved these changes Nov 30, 2022

View reviewed changes

chensuyue reviewed Nov 30, 2022

View reviewed changes

xin3he approved these changes Nov 30, 2022

View reviewed changes

chensuyue added examples extension test labels Nov 30, 2022

chensuyue added this to the v2.0 milestone Nov 30, 2022

PenghuiCheng and others added 17 commits December 1, 2022 21:45

Set default value for use_bf16 and fixed random seed setting error (#186

83825af

) Signed-off-by: Cheng, Penghui <[email protected]>

Turn off ITEX optimization pass (#196)

df8c5f4

Signed-off-by: Lv, Liang1 <[email protected]>

update example json

fb5560e

Signed-off-by: changwa1 <[email protected]>

Fixed UT error for bf16 op list for QAT mode (#200)

694f22b

* Fixed UT error for bf16 op list for QAT mode Signed-off-by: Cheng, Penghui <[email protected]>

Disable multi instance for ITEX GPU benchmark (#204)

5d22e01

Signed-off-by: Lv, Liang1 <[email protected]>

Revert "remove op-wise cfgs for testing. (#1521)" (#202)

0346c53

Signed-off-by: intel-zhangyi <[email protected]>

add examples for GPTJ (#162)

01899d6

Signed-off-by: Wang, Chang1 <[email protected]>

Neural Coder mod launcher arg: "strategy" to "approach" (#201)

b48ff81

update publication_list.md (#212)

5855117

Signed-off-by: chensuyue <[email protected]>

Added distributed training support for distillation of CNN-2. (#208)

ebe9e2a

Signed-off-by: Xinyu Ye <[email protected]>

Added distributed training support for distillation of MobileNetV2. (#…

d33ebe6

…166) Signed-off-by: Xinyu Ye <[email protected]>

fix load issue (#194)

08fe8dd

Signed-off-by: changwa1 <[email protected]>

Fix NTM-One-Shot failed with KeyError (#210)

e9be412

Signed-off-by: Lv, Liang1 <[email protected]> Signed-off-by: Lv, Liang1 <[email protected]>

Neural Coder launcher debug (#215)

895300a

* update launcher to fit multi-item input * Update __main__.py

Neural Coder enable support of TensorFlow/Keras models to quantize (p…

d28bd14

…reliminary) (#217) * Create tf_inc_static_quant.yaml * Create inc.py * Delete tf_inc_static_quant.yaml * Update interface.py * Update inc.py

XuehaoSun and others added 24 commits December 9, 2022 22:26

azure ut coverage report fix (#252)

c61be34

Signed-off-by: Sun, Xuehao <[email protected]>

Refactor Quantization Aware Training of TF backend (#250)

1deb7d2

Signed-off-by: zehao-intel <[email protected]>

update publications (#255)

8b652cd

Signed-off-by: chensuyue <[email protected]>

Add hawq_v2 tuning strategy (#230)

83018ef

Signed-off-by: yiliu30 <[email protected]> Co-authored-by: lvliang-intel <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: xinhe <[email protected]> Co-authored-by: Ray <[email protected]>

Fixed pruning and distillation bug and remove invalid code (#251)

a230726

Signed-off-by: Cheng, Penghui <[email protected]>

add keras-in/keras-out to INC (#243)

4fa7531

Signed-off-by: Lv, Liang1 <[email protected]> Signed-off-by: chensuyue <[email protected]> Co-authored-by: Lv, Liang1 <[email protected]> Co-authored-by: chensuyue <[email protected]>

add warning when meets unsupported config (#236)

c53e403

Signed-off-by: Xin He <[email protected]> Co-authored-by: yiliu30 <[email protected]>

Remove data, metric and common to neural_compressor (#244)

30803cf

Signed-off-by: changwa1 <[email protected]>

CI UT enhance (#258)

a18ff5c

Signed-off-by: chensuyue <[email protected]>

Neural Coder enable launcher bench (#260)

fcbbcc7

* enable auto bench in launcher * debug * debug

Enable Transformer LT search space for Dynamic Neural Architecture Se…

40ab5a3

…arch Toolkit (#197) Signed-off-by: Maciej Szankin <[email protected]> Co-authored-by: Nittur Sridhar, Sharath <[email protected]> Co-authored-by: Xinyu Ye <[email protected]>

Export Qlinear to QDQ (#224)

e996a93

Signed-off-by: mengniwa <[email protected]>

Update spr-base version number (#259)

cde72c8

Signed-off-by: Lv, Liang1 <[email protected]> Co-authored-by: chen, suyue <[email protected]>

Fixed calibration sampling size error and IPEX examples error (#264)

ae3cf56

* Fixed calibration sampling size error * Update training fit API docstring * Fixed IPEX examples error Signed-off-by: Cheng, Penghui <[email protected]>

Enhancement benchmark with dataloader (#269)

f21e4a3

Signed-off-by: Cheng, Penghui <[email protected]>

Fix TF QAT UT issues (#266)

7d1e1f9

Signed-off-by: zehao-intel <[email protected]>

Add recipe for TRT EP (#278)

764357b

* Add recipe for TRT EP Signed-off-by: Mengni Wang <[email protected]> * remove codes Signed-off-by: Mengni Wang <[email protected]> Signed-off-by: Mengni Wang <[email protected]>

Refine Keras Examples for INC New API (#219)

0878bea

Signed-off-by: zehao-intel <[email protected]>

update quantization api

e606b02

Signed-off-by: changwa1 <[email protected]>

update example json

6c605c1

Signed-off-by: changwa1 <[email protected]>

add benchmark and fx+static for torchaudio

ecc6e8b

fix conflict

2d06c22

changwangss closed this Dec 15, 2022

chensuyue deleted the wangchang/example branch June 15, 2023 05:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update PyTorch examples quantization API #192

Update PyTorch examples quantization API #192

Uh oh!

changwangss commented Nov 30, 2022 •

edited

Loading

Uh oh!

chensuyue Nov 30, 2022

Uh oh!

chensuyue Nov 30, 2022

Uh oh!

chensuyue commented Nov 30, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

24 participants

Update PyTorch examples quantization API #192

Update PyTorch examples quantization API #192

Uh oh!

Conversation

changwangss commented Nov 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type of Change

Description

Expected Behavior & Potential Risk

How has this PR been tested?

Dependency Change?

Uh oh!

chensuyue Nov 30, 2022

Choose a reason for hiding this comment

Uh oh!

chensuyue Nov 30, 2022

Choose a reason for hiding this comment

Uh oh!

chensuyue commented Nov 30, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

24 participants

changwangss commented Nov 30, 2022 •

edited

Loading