You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Intel® Low Precision Optimization Tool is an open-source python library which is intended to deliver a unified low-precision inference interface cross multiple Intel optimized DL frameworks on both CPU and GPU. It supports automatic accuracy-driven tuning strategies, along with additional objectives like performance, model size, or memory footprint. It also provides the easy extension capability for new backends, tuning strategies, metrics and objectives.
5
5
6
+
6
7
> **WARNING**
7
8
>
8
9
> GPU support is under development.
@@ -16,13 +17,19 @@ Currently supported tuning strategies are:
[Introduction](docs/introduction.md) explains Intel® Low Precision Optimization Tool infrastructure, design philosophy, supported functionality, details of tuning strategy implementations and tuning result on popular models.
22
27
23
-
*[Introduction](docs/introduction.md) explains Intel® Low Precision Optimization Tool infrastructure, design philosophy, supported functionality, details of tuning strategy implementations and tuning result on popular models.
24
-
*[Tutorial](docs/tutorial.md) provides
25
-
comprehensive step-by-step instructions of how to enable Intel® Low Precision Optimization Tool on sample models.
28
+
# Tutorials
29
+
*[Hello World](examples/helloworld/README.md) demonstrates the simple steps to utilize Intel® Low Precision Optimization Tool for quanitzation, which can help you quick start with the tool.
30
+
*[Tutorials](docs/README.md) provides
31
+
comprehensive instructions of how to utilize diffrennt features of Intel® Low Precision Optimization Tool.
32
+
*[Examples](examples) is a tuning zoo to demonstrate the usage of Intel® Low Precision Optimization Tool in TensorFlow, PyTorch and MxNet for industry models of diffrent categories.
26
33
27
34
# Install from source
28
35
@@ -57,21 +64,59 @@ Intel® Low Precision Optimization Tool requires to install Intel optimized fram
57
64
58
65
The followings are the examples integrated with Intel® Low Precision Optimization Tool for auto tuning.
59
66
60
-
| Model | Framework | Model | Framework | Model | Framework |
Copy file name to clipboardExpand all lines: docs/introduction.md
+36-5Lines changed: 36 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,15 +22,38 @@ Intel® Low Precision Optimization Tool is an open source python library to help
22
22
23
23
The tuning config and model-specific information are controlled by user config yaml file. As for the format of yaml file, please refer to [template.yaml](../examples/template.yaml)
24
24
25
-
Intel® Low Precision Optimization Tool supports two usages:
25
+
Intel® Low Precision Optimization Tool supports three usages:
26
26
27
-
a) User specifies fp32 "model", calibration dataset "q_dataloader", evaluation dataset "eval_dataloader" and accuracy metrics in tuning.metric field of the yaml config file.
27
+
a) Fully yaml configuration: User specifies all the info through yaml, including dataloaders used in calibration and evaluation
28
+
phases and quantization tuning settings.
29
+
30
+
For this usage, only model parameter is mandotory.
31
+
32
+
b) Partial yaml configuration: User specifies dataloaders used in calibration and evaluation phase by code.
33
+
The tool provides built-in dataloaders and evaluators, user just need provide a dataset implemented __iter__ or
34
+
__getitem__ methods and invoke dataloader() with dataset as input parameter before calling tune().
35
+
36
+
After that, User specifies fp32 "model", calibration dataset "q_dataloader" and evaluation dataset "eval_dataloader".
37
+
The calibrated and quantized model is evaluated with "eval_dataloader" with evaluation metrics specified
38
+
in the configuration file. The evaluation tells the tuner whether the quantized model meets
39
+
the accuracy criteria. If not, the tuner starts a new calibration and tuning flow.
40
+
41
+
For this usage, model, q_dataloader and eval_dataloader parameters are mandotory.
42
+
43
+
c) Partial yaml configuration: User specifies dataloaders used in calibration phase by code.
44
+
This usage is quite similar with b), just user specifies a custom "eval_func" which encapsulates
45
+
the evaluation dataset by itself.
46
+
The calibrated and quantized model is evaluated with "eval_func". The "eval_func" tells the
47
+
tuner whether the quantized model meets the accuracy criteria. If not, the Tuner starts a new
48
+
calibration and tuning flow.
49
+
50
+
For this usage, model, q_dataloader and eval_func parameters are mandotory
28
51
29
-
b) User specifies fp32 "model", calibration dataset "q_dataloader" and a custom "eval_func" which encapsulates the evaluation dataset and accuracy metrics by itself.
52
+
The first usage is designed for minimal code changes when integrating with Intel® Low Precision Optimization Tool. All calibration and evaluation process is constructed by ilit upon yaml configuration.
30
53
31
-
The first usage is designed for seamless enablement of DL model tuning with Intel® Low Precision Optimization Tool, leveraging the pre-defined accuracy metrics supported by this tool. We expect this is the most common usage. Now it works well for most image classification models and we are improving the tool to support more.
54
+
The second usage is designed for concise yaml configuration and constructing calibration and evaluation dataloaders by code. ilit provides built-in dataloaders and evaluators, user just need provide a dataset implemented __iter__ or __getitem__ methods to tuner.dataloader() function.
32
55
33
-
The second usage is designed for ease of tuning enablement for models with custom metric evaluation or metrics not supported by Intel® Low Precision Optimization Tool yet. Currently this usage model works for object detection and NLP networks.
56
+
The third usage is designed for ease of tuning enablement for models with custom metric evaluation or metrics not supported by Intel® Low Precision Optimization Tool yet. Currently this usage model works for object detection and NLP networks.
34
57
35
58
2. Framework Adaptation API
36
59
@@ -60,6 +83,14 @@ Bayesian optimization is a sequential design strategy for global optimization of
60
83
61
84
This strategy is very similar to the basic strategy. It needs to get the tensors for each Operator of raw FP32 models and the quantized model based on best model-wise tuning configuration. And then calculate the MSE (Mean Squared Error) for each operator, sort those operators according to the MSE value, finally do the op-wise fallback in this order.
62
85
86
+
### Random Strategy
87
+
88
+
This strategy is used to randomly choose tuning configuration from the tuning space.
89
+
90
+
### Exhaustive Strategy
91
+
92
+
This strategy is used to sequentially traverse all the possible tuning configurations in tuning space.
93
+
63
94
# Objectives
64
95
65
96
Intel® Low Precision Optimization Tool supports below 3 build-in objectives. All objectives are optimized and driven by accuracy metrics.
0 commit comments