Rocm mx gemm #5
Open
Rocm mx gemm #5
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…dation logic. Added MXFP8_HIPBLASLT recipe and adjusted mx_mm function to accommodate new kernel options.
…ASLT kernel choice for mxfp8 gemm. Enhance documentation on end-to-end performance optimization efforts for AMD GPUs.
…py to include HIPBLASLT as a valid kernel choice for MX FP8 operations.
Owner
Author
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Bug: Outdated Assertion Message for MX FP8 Operations
The assertion error message "CUBLAS is the only supported kernel choice for MX FP8 operations" is incorrect. The code's assertion logic now accepts both CUBLAS and HIPBLASLT kernel choices for MX FP8 operations. This outdated message misleads users about supported kernel choices and should be updated.
torchao/prototype/mx_formats/mx_ops.py#L109-L113
ao/torchao/prototype/mx_formats/mx_ops.py
Lines 109 to 113 in 129a6d6
BugBot free trial expires on June 16, 2025
You have used $0.00 of your $20.00 spend limit so far. Manage your spend limit in the Cursor dashboard.
Was this report helpful? Give feedback by reacting with 👍 or 👎
… HIPBLASLT are supported kernel choices for MX FP8 operations.
…l choices for MX FP8 operations.
Co-authored-by: Copilot <[email protected]>
- Introduced `is_ROCm_mx_supported` function to verify ROCm environment compatibility for MX operations. - Added `test_hipblaslt_fp8` to validate FP8 operations using the HIPBLASLT backend, including SQNR verification for output accuracy. - Updated imports in `test_mx_mm.py` to include necessary utilities for the new test.
- Replaced `compute_sqnr` with `compute_error` for improved accuracy in error measurement. - Updated assertion to ensure output accuracy meets the specified threshold.
- Updated the function to ensure `torch.version.hip` is not None before checking the version, improving robustness against potential NoneType errors.
- Reformatted the return statement to enhance clarity and maintainability of the code.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces support for AMD MI355x GPUs with ROCm 6.5+ and HIPBLASLT for MX gemm operations, alongside updates to the documentation and validation logic. The changes expand the framework's compatibility and functionality for AMD hardware while maintaining support for existing NVIDIA configurations.
AMD Support Enhancements:
torchao/prototype/mx_formats/config.py:HIPBLASLTas a new kernel choice inMXGemmKernelChoiceandMXFP8_HIPBLASLTinMXLinearRecipeName._validate_gemm_kernel_choiceto include validation forHIPBLASLT, ensuring specific block size, data types, and ROCm availability.from_recipe_nameto handleMXFP8_HIPBLASLT.torchao/prototype/mx_formats/mx_ops.py:_addmm_mx_dispatchto supportHIPBLASLTfor matrix-matrix operations in addition to existing kernel choices. [1] [2]Documentation Updates:
torchao/prototype/mx_formats/README.md: