roofline estimator: add float8 rowwise and mxfp8 recipe support #1789

vkuzo · 2025-02-26T23:59:28Z

Summary:

Various changes to beef up float8 and mx roofline estimations.

Test Plan:

https://gist.github.com/vkuzo/786f4d3aef2a042ca4d8ef40dbd35d64
https://gist.github.com/vkuzo/75c46bb4b0f5bf0672223230c8dda78a


python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise
python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]

vkuzo · 2025-02-26T23:59:29Z

Stack from ghstack (oldest at bottom):

-> roofline estimator: add float8 rowwise and mxfp8 recipe support #1789

pytorch-bot · 2025-02-26T23:59:31Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1789

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit 9bedb01 with merge base b9c51b7 ():

NEW FAILURES - The following jobs have failed:

Run Regression Tests / test (CUDA 2.4, linux.g5.12xlarge.nvidia.gpu, torch==2.4.0, cuda, 12.1) / linux-job (gh)
RuntimeError: Command docker exec -t 54b7b7e40dca6c3f5ab9fcc26332376488bc55a7df8e7cfe963e6e5788b61b70 /exec failed with exit code 1
Run Regression Tests / test (CUDA 2.5.1, linux.g5.12xlarge.nvidia.gpu, torch==2.5.1 --index-url https://download.pytorch... / linux-job (gh)
RuntimeError: Command docker exec -t c13d90e62e0a3f3947b639e1eac827f7baf8acb9b8ec92b477e30a2ddf1cf923 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Summary: Test Plan: ``` python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 01f4cf1 ghstack-comment-id: 2686473047 Pull Request resolved: #1789

[ghstack-poisoned]

Summary: Test Plan: ``` python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 01f4cf1 ghstack-comment-id: 2686473047 Pull Request resolved: #1789

[ghstack-poisoned]

Summary: Test Plan: ``` python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 01f4cf1 ghstack-comment-id: 2686473047 Pull Request resolved: #1789

[ghstack-poisoned]

Summary: Test Plan: ``` python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 01f4cf1 ghstack-comment-id: 2686473047 Pull Request resolved: #1789

[ghstack-poisoned]

Summary: Test Plan: ``` python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 01f4cf1 ghstack-comment-id: 2686473047 Pull Request resolved: #1789

[ghstack-poisoned]

Summary: Test Plan: ``` python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 9b83266 ghstack-comment-id: 2686473047 Pull Request resolved: #1789

[ghstack-poisoned]

Summary: Test Plan: ``` python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 19c2e7c ghstack-comment-id: 2686473047 Pull Request resolved: #1789

[ghstack-poisoned]

Summary: Test Plan: ``` python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: a34af93 ghstack-comment-id: 2686473047 Pull Request resolved: #1789

[ghstack-poisoned]

Summary: Test Plan: ``` python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: fbc134f ghstack-comment-id: 2686473047 Pull Request resolved: #1789

[ghstack-poisoned]

Summary: Test Plan: ``` python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 7857b5c ghstack-comment-id: 2686473047 Pull Request resolved: #1789

[ghstack-poisoned]

Summary: Test Plan: ``` python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --float8_recipe_name rowwise python benchmarks/float8/float8_roofline.py ~/local/tmp/20250226_test.csv --n_limit 1 --mx_recipe_name mxfp8_emulated ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: a85a625 ghstack-comment-id: 2686473047 Pull Request resolved: #1789

* Update [ghstack-poisoned] * Update [ghstack-poisoned] * Update [ghstack-poisoned] * Update [ghstack-poisoned] * Update [ghstack-poisoned] * Update [ghstack-poisoned] * Update [ghstack-poisoned] * Update [ghstack-poisoned] * Update [ghstack-poisoned] * Update [ghstack-poisoned]

vkuzo added 13 commits February 25, 2025 14:03

Update

5b01edc

[ghstack-poisoned]

Update

0ae7f46

[ghstack-poisoned]

Update

77c8628

[ghstack-poisoned]

Update

086ff72

[ghstack-poisoned]

Update

00fb1b8

[ghstack-poisoned]

Update

64647f1

[ghstack-poisoned]

Update

4bed185

[ghstack-poisoned]

Update

2f94635

[ghstack-poisoned]

Update

73eec2e

[ghstack-poisoned]

Update

ac27fdd

[ghstack-poisoned]

Update

acc907a

[ghstack-poisoned]

Update

e9ab762

[ghstack-poisoned]

Update

b571ba8

[ghstack-poisoned]

This was referenced Feb 26, 2025

[1/x] mx roofline: make the script work on NVIDIA B200 #1778

Merged

roofline estimation: delete scaling type #1781

Merged

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 26, 2025

This was referenced Feb 26, 2025

roofline estimation: delete axiswise scaling, for now #1782

Merged

roofline estimator: simplify #1783

Merged

vkuzo added 4 commits February 27, 2025 08:12

Update

bf83a68

[ghstack-poisoned]

Update

400158b

[ghstack-poisoned]

Update

fa6d2e5

[ghstack-poisoned]

Update

8d96899

[ghstack-poisoned]

vkuzo added 3 commits February 27, 2025 08:13

Update

aa0abab

[ghstack-poisoned]

Update

6828261

[ghstack-poisoned]

Update

84da346

[ghstack-poisoned]

vkuzo added 2 commits February 27, 2025 08:14

Update

7de3541

[ghstack-poisoned]

Update

86486bd

[ghstack-poisoned]

Update

83cd4a9

[ghstack-poisoned]

vkuzo changed the base branch from gh/vkuzo/45/head to main February 27, 2025 16:15

Update

c1a7bad

[ghstack-poisoned]

Update

0786de8

[ghstack-poisoned]

Update

dfe06b8

[ghstack-poisoned]

Update

ea1d121

[ghstack-poisoned]

vkuzo added the topic: not user facing Use this tag if you don't want this PR to show up in release notes label Feb 28, 2025

Update

6a5718d

[ghstack-poisoned]

Update

9bedb01

[ghstack-poisoned]

jcaip approved these changes Mar 4, 2025

View reviewed changes

vkuzo merged commit d8af7d7 into main Mar 4, 2025
44 of 47 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

roofline estimator: add float8 rowwise and mxfp8 recipe support #1789

roofline estimator: add float8 rowwise and mxfp8 recipe support #1789

Uh oh!

vkuzo commented Feb 26, 2025 •

edited

Loading

Uh oh!

vkuzo commented Feb 26, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Feb 26, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

roofline estimator: add float8 rowwise and mxfp8 recipe support #1789

roofline estimator: add float8 rowwise and mxfp8 recipe support #1789

Uh oh!

Conversation

vkuzo commented Feb 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vkuzo commented Feb 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Feb 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1789

❌ 2 New Failures

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vkuzo commented Feb 26, 2025 •

edited

Loading

vkuzo commented Feb 26, 2025 •

edited

Loading

pytorch-bot bot commented Feb 26, 2025 •

edited

Loading