Skip to content

Conversation

Hariom-Nagar211
Copy link

Summary

Adds ONNX Runtime static quantization support for deblurring_nafnet, and a validation script to compare FP32 vs INT8 outputs with PSNR/SSIM and timing.
Improves quantization robustness and memory handling across the tools.
Changes

tools/quantize/quantize-ort.py
Adds deblurring_nafnet entry to the central models dict.
Uses QuantFormat.QOperator + op_types_to_quantize=['Conv','MatMul'] for better CPU EP performance, with an automatic fallback to QDQ Conv-only when needed.
Optional pre-processing (skips if onnxruntime-extensions is missing).
Lazy
DataReader
creation to avoid importing all datasets at module import.
MinMax calibration; input resize to 512x512; max_samples=1 to reduce memory.
Auto-excludes Conv nodes without bias initializers to avoid bias=None errors.
tools/quantize/transform.py
Makes
HandAlign
lazy-load its palm detector so unrelated quant runs don’t load Mediapipe ONNX at import time.
models/deblurring_nafnet/validate_quantization.py
New: Runs FP32 and INT8 models via ORT, reports timing, PSNR, SSIM, and shows side-by-side results.
Usage

Quantize NafNet:
cd tools/quantize
python quantize-ort.py deblurring_nafnet
Output: models/deblurring_nafnet/deblurring_nafnet_2025may_int8.onnx
Validate:
cd models/deblurring_nafnet
python validate_quantization.py --input example_outputs/licenseplate_motion.jpg --model_fp32 deblurring_nafnet_2025may.onnx --model_int8 deblurring_nafnet_2025may_int8.onnx
Notes on performance

On CPUExecutionProvider, INT8 speed-ups are not guaranteed unless the provider fuses/accelerates int8 kernels.
For acceleration, consider provider-specific EPs:
OpenVINOExecutionProvider (CPU) or DmlExecutionProvider (GPU) if available.
The script quantizes to QOperator Conv/MatMul by default; falls back to QDQ Conv-only for robustness.
Known limitations

Quantization depends on ORT support for the model’s ops and shapes; the script includes automatic mitigations (resize, sample limit, excludes).
We do not commit generated ONNX or output images.

Screenshot:
Screenshot 2025-09-22 213940

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant