[mlir][tosa] Add support for mxint8 type in mxfp operations #163642
+2,064
−439
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds a new "specification_version" field to the TOSA target environment attribute. This allows a user to specify which version of the TOSA specification they would like to target during lowering. A leading example in the validation pass has also been added. This addition adds a version to each profile compliance entry to track which version of the specification the entry was added. This allows a backwards compatibility check to be implemented between the target version and the profile compliance entry version. For now a default version of "1.0" is assumed. "1.1.draft" is added to denote an in-development version of the specification targeting the next release. Change-Id: I6549e05bd4fe975d12ea31e8acc783233db66171
This commit adds support for the MATMUL_T_BLOCK_SCALED operation from the EXT_MXFP extension. This includes: - Operation definition in TosaOps.td - Micro-scaling supported types definition - Shape inference and verifiers - Validation pass checks to ensure usage is only valid when the target environment includes ext-mxfp and at least v1.1.draft of the specification. As part of this commit, a notion of EXT_MXFP is also added. The extension can be specified as part of the target environment and can only be used if the specification version is at least 1.1. Note: currently it excludes support for mxint8. This will be added in a later commit. Note: this commit adds support as defined in the spec in https://review.mlplatform.org/c/tosa/specification/+/15362. EXT_MXFP extension is considered experimental and subject to breaking change. Co-authored-by: Tat Wai Chong <[email protected]> Change-Id: I92afdea87eef1eea444dfebf9f74796f3a236809
This commit adds support for the cast_from/to_block_scaled operations from the ext-mxfp extension. This includes: - Operation definition in TosaOps.td - Micro-scaling supported types definition - Shape inference and verifiers - Validation pass checks to ensure usage is only valid when the target environment includes ext-mxfp and at least v1.1.draft of the specification. Note: currently it excludes support for mxint8. This will be added in a later commit. Note: this commit adds support as defined in the spec in https://review.mlplatform.org/c/tosa/specification/+/15362. EXT_MXFP extension is considered experimental and subject to breaking change. Co-authored-by: Tat Wai Chong <[email protected]> Change-Id: I490645ce99b7ccd7021ed06acaf1530b4fbf6dfd
This commit allows the creation of const/cast operations with MXFP datatypes. Note: it doesn't include support for the mxint8 datatype. This will be added in a separate commit. Note: this commit adds support as defined in the spec in https://review.mlplatform.org/c/tosa/specification/+/15362. EXT_MXFP extension is considered experimental and subject to breaking change. Change-Id: Idd0477bc947ade524b0fb0213cc7e8d4f892ddab
This commit adds support for the OCP-MX INT8 type. This includes the following operations: MATMUL_T_BLOCK_SCALED, CAST_FROM_BLOCK_SCALED, CAST_TO_BLOCK_SCALED and CONST. The support is added via a custom TOSA type "!tosa.mxint8" due to the fact it is not yet a builtin type in mlir. This may change in the future, depending on how this type is used by other frameworks/ dialects. Conversions to/from this type have not yet been implemented for the same reasoning. Co-authored-by: Tat Wai Chong <[email protected]> Change-Id: I6dbba8d55075111cae6b3186cef90fd87d9e5ae6
You can test this locally with the following command:git-clang-format --diff origin/main HEAD --extensions cpp,h,inc -- mlir/include/mlir/Dialect/Tosa/IR/TargetEnv.h mlir/include/mlir/Dialect/Tosa/IR/TosaComplianceData.h.inc mlir/include/mlir/Dialect/Tosa/IR/TosaOps.h mlir/include/mlir/Dialect/Tosa/IR/TosaProfileCompliance.h mlir/lib/Dialect/Tosa/IR/TargetEnv.cpp mlir/lib/Dialect/Tosa/IR/TosaOps.cpp mlir/lib/Dialect/Tosa/Transforms/TosaAttachTarget.cpp mlir/lib/Dialect/Tosa/Transforms/TosaProfileCompliance.cpp mlir/lib/Dialect/Tosa/Transforms/TosaValidation.cpp --diff_from_common_commit
View the diff from clang-format here.diff --git a/mlir/include/mlir/Dialect/Tosa/IR/TosaComplianceData.h.inc b/mlir/include/mlir/Dialect/Tosa/IR/TosaComplianceData.h.inc
index 2d3ff2f28..3a36806a3 100644
--- a/mlir/include/mlir/Dialect/Tosa/IR/TosaComplianceData.h.inc
+++ b/mlir/include/mlir/Dialect/Tosa/IR/TosaComplianceData.h.inc
@@ -556,12 +556,18 @@ extensionComplianceMap = {
{{{bf16T, bf16T, bf16T, bf16T, fp32T}, SpecificationVersion::V_1_0}}}}},
{"tosa.matmul_t_block_scaled",
{{{Extension::mxfp},
- {{{fp4e2m1T, fp8ue8m0T, fp4e2m1T, fp8ue8m0T, fp32T}, SpecificationVersion::V_1_1_DRAFT},
- {{fp6e2m3T, fp8ue8m0T, fp6e2m3T, fp8ue8m0T, fp32T}, SpecificationVersion::V_1_1_DRAFT},
- {{fp6e3m2T, fp8ue8m0T, fp6e3m2T, fp8ue8m0T, fp32T}, SpecificationVersion::V_1_1_DRAFT},
- {{fp8e4m3T, fp8ue8m0T, fp8e4m3T, fp8ue8m0T, fp32T}, SpecificationVersion::V_1_1_DRAFT},
- {{fp8e5m2T, fp8ue8m0T, fp8e5m2T, fp8ue8m0T, fp32T}, SpecificationVersion::V_1_1_DRAFT},
- {{mxint8T, fp8ue8m0T, mxint8T, fp8ue8m0T, fp32T}, SpecificationVersion::V_1_1_DRAFT}}}}},
+ {{{fp4e2m1T, fp8ue8m0T, fp4e2m1T, fp8ue8m0T, fp32T},
+ SpecificationVersion::V_1_1_DRAFT},
+ {{fp6e2m3T, fp8ue8m0T, fp6e2m3T, fp8ue8m0T, fp32T},
+ SpecificationVersion::V_1_1_DRAFT},
+ {{fp6e3m2T, fp8ue8m0T, fp6e3m2T, fp8ue8m0T, fp32T},
+ SpecificationVersion::V_1_1_DRAFT},
+ {{fp8e4m3T, fp8ue8m0T, fp8e4m3T, fp8ue8m0T, fp32T},
+ SpecificationVersion::V_1_1_DRAFT},
+ {{fp8e5m2T, fp8ue8m0T, fp8e5m2T, fp8ue8m0T, fp32T},
+ SpecificationVersion::V_1_1_DRAFT},
+ {{mxint8T, fp8ue8m0T, mxint8T, fp8ue8m0T, fp32T},
+ SpecificationVersion::V_1_1_DRAFT}}}}},
{"tosa.max_pool2d",
{{{Extension::int16}, {{{i16T, i16T}, SpecificationVersion::V_1_0}}},
{{Extension::fp8e4m3},
@@ -755,7 +761,8 @@ extensionComplianceMap = {
{{fp6e2m3T, bf16T}, SpecificationVersion::V_1_1_DRAFT},
{{bf16T, fp4e2m1T}, SpecificationVersion::V_1_1_DRAFT},
{{bf16T, fp6e3m2T}, SpecificationVersion::V_1_1_DRAFT},
- {{bf16T, fp6e2m3T}, SpecificationVersion::V_1_1_DRAFT}}, allOf}}},
+ {{bf16T, fp6e2m3T}, SpecificationVersion::V_1_1_DRAFT}},
+ allOf}}},
{"tosa.cast_from_block_scaled",
{{{Extension::bf16, Extension::mxfp},
{{{fp4e2m1T, fp8ue8m0T, bf16T}, SpecificationVersion::V_1_1_DRAFT},
@@ -763,7 +770,8 @@ extensionComplianceMap = {
{{fp6e3m2T, fp8ue8m0T, bf16T}, SpecificationVersion::V_1_1_DRAFT},
{{fp8e4m3T, fp8ue8m0T, bf16T}, SpecificationVersion::V_1_1_DRAFT},
{{fp8e5m2T, fp8ue8m0T, bf16T}, SpecificationVersion::V_1_1_DRAFT},
- {{mxint8T, fp8ue8m0T, bf16T}, SpecificationVersion::V_1_1_DRAFT}}, allOf},
+ {{mxint8T, fp8ue8m0T, bf16T}, SpecificationVersion::V_1_1_DRAFT}},
+ allOf},
{{Extension::mxfp},
{{{fp4e2m1T, fp8ue8m0T, fp32T}, SpecificationVersion::V_1_1_DRAFT},
{{fp6e2m3T, fp8ue8m0T, fp32T}, SpecificationVersion::V_1_1_DRAFT},
@@ -785,7 +793,8 @@ extensionComplianceMap = {
{{bf16T, fp6e3m2T, fp8ue8m0T}, SpecificationVersion::V_1_1_DRAFT},
{{bf16T, fp8e4m3T, fp8ue8m0T}, SpecificationVersion::V_1_1_DRAFT},
{{bf16T, fp8e5m2T, fp8ue8m0T}, SpecificationVersion::V_1_1_DRAFT},
- {{bf16T, mxint8T, fp8ue8m0T}, SpecificationVersion::V_1_1_DRAFT}}, allOf}}},
+ {{bf16T, mxint8T, fp8ue8m0T}, SpecificationVersion::V_1_1_DRAFT}},
+ allOf}}},
{"tosa.rescale",
{{{Extension::int16},
{{{i48T, i48T, i8T, i8T}, SpecificationVersion::V_1_0},
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds support for the OCP-MX INT8 type. This includes the following operations: MATMUL_T_BLOCK_SCALED, CAST_FROM_BLOCK_SCALED, CAST_TO_BLOCK_SCALED and CONST.
The support is added via a custom TOSA type "!tosa.mxint8" due to the fact it is not yet a builtin type in mlir. This may change in the future, depending on how this type is used by other frameworks/dialects. Conversions to/from this type have not yet been implemented for the same reasoning.
Note: This PR relies on #156425, #163433, #163436 and #163641 so also contains their contents.
Co-authored-by: Tat Wai Chong [email protected]