support tuning target_bits

[xin3he]

User description

Type of Change

Feature

Description

Feature: support AutoRound target_bits and autotune bits.

Expected Behavior & Potential Risk

• Automatically generate mix-precision model
• Autotune target_bits based on the evaluation result.

How has this PR been tested?

UT pass

Dependency Change?

AutoRound >= 0.9.0

---

PR Type

Enhancement

---

Description

• Add support for tuning target_bits

• Introduce new parameters for auto scheme configuration

• Update preprocessing in autotune function

---

Diagram Walkthrough

flowchart LR
  A["Add target_bits"] -- "New parameter" --> B["Update __init__"]
  B -- "Include new parameters" --> C["Modify autotune"]
  C -- "Preprocess model and config" --> D["Enhance AutoRoundConfig"]

Loading

File Walkthrough

	Relevant files
Enhancement	3 files config.py Add target_bits and auto scheme parameters                              +41/-4    autotune.py Preprocess model and quant config                                                +13/-1    base_config.py Update parameter handling                                                                +3/-3
Additional files	4 files tuning_param.py +1/-1      autoround.py +57/-1    algorithm_entry.py +16/-1    test_autoround.py +75/-4

---

[PRAgent4INC]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Default Value The target_bits parameter is set to None by default, which might lead to unexpected behavior if not explicitly set by the user. options: Union[str, list[Union[str]], tuple[Union[str], ...]] = ("MXFP4", "MXFP8"), Import Statement The import statement for AutoScheme is inside the convert method, which can lead to increased load times and potential circular import issues. if self.target_bits is not None: Hardcoded Op Type The dump_model_op_stats function currently only collects statistics for the "Linear" op type. This might need to be extended to support other types of operations. """Dump quantizable ops stats of model to user.""" # TODO: collect more ops besides Linear res = {}

[PRAgent4INC]

PR Code Suggestions ✨

[Copilot]

Pull Request Overview

This PR adds support for tuning target_bits in the AutoRound quantization configuration, enabling automatic mixed-precision model generation and autotuning of target bits based on evaluation results.

Key changes:

• Added new target_bits parameter and auto scheme configuration options to AutoRoundConfig
• Implemented preprocessing of model and quantization config in the autotune function to handle tokenizer attributes
• Updated parameter handling in base_config.py to use type annotations instead of default values

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
config.py	Added target_bits and related auto scheme parameters to AutoRoundConfig for mixed-precision support
autotune.py	Introduced preprocessing function to handle tokenizer attributes before quantization
base_config.py	Modified parameter expansion logic to use type annotations
tuning_param.py	Moved model creation inside try-catch block for better error handling
autoround.py	Implemented auto scheme creation when target_bits is set and added statistics dumping
algorithm_entry.py	Passed new auto scheme parameters to the quantizer
test_autoround.py	Added tests for target_bits functionality and moved imports to module level

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[yiliu30]

LGTM