Skip to content

Conversation

@xin3he
Copy link
Contributor

@xin3he xin3he commented Dec 12, 2023

Type of Change

bug fix

Description

IPEX backend got wrong number of quantizable add ops when detecting.
Quantizable add ops should be detected in gptj but not in opt model.

Expected Behavior & Potential Risk

add ops can only be detected if both input inf_dtypes are not float32.

        "q_op_infos": {
            "0": {
                "op_type": "<method 'add' of 'torch._C._TensorBase' objects>",
                "op_type_is_module": false,
                "fqn": "",
                "input_tensor_infos": [
                    {
                        "id": 0,
                        "orig_dtype": "torch.float32",
                        "inf_dtype": "torch.float32",
                        "force_dtype": "torch.float32",
                        "scale": [
                            0.007949741557240486
                        ],
                        "zero_point": [
                            219
                        ]
                    },
                    {
                        "id": 0,
                        "orig_dtype": "torch.float32",
                        "inf_dtype": "torch.float32",
                        "force_dtype": "torch.float32",
                        "scale": [
                            0.007949741557240486
                        ],
                        "zero_point": [
                            219
                        ]
                    }
                ],

How has this PR been tested?

local test on models

Dependency Change?

N/A

Signed-off-by: Xin He <[email protected]>
Signed-off-by: Xin He <[email protected]>
Signed-off-by: Xin He <[email protected]>
@xin3he xin3he changed the title Xinhe/ipex [bug fix] update quantizable add ops detection in IPEX backend Dec 12, 2023
@xin3he xin3he changed the title [bug fix] update quantizable add ops detection in IPEX backend [bug fix] update quantizable add ops detection on IPEX backend Dec 12, 2023
@chensuyue chensuyue added the bug fix Something isn't working label Dec 12, 2023
@chensuyue chensuyue modified the milestones: v2.4, v2.5, v2.4.1 Dec 12, 2023
@chensuyue chensuyue merged commit 4c004d7 into master Dec 25, 2023
@chensuyue chensuyue deleted the xinhe/ipex branch December 25, 2023 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug fix Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants