[E2E] Introduce SPIR-V Backend support in SYCL tests and end-to-end tests #16317

VyacheslavLevytskyy · 2024-12-10T11:51:17Z

Introduce SPIR-V Backend support in SYCL tests and end-to-end tests by adding XFAIL & UNSUPPORTED to tests that are not passed at the moment when Khronos Translator is substituted by SPIR-V Backend. Three syntax-only tests are moved from e2e to tests.

VyacheslavLevytskyy · 2024-12-10T12:04:32Z

@AlexeySachkov @sarnex Please have a look. This PR shows how we possibly can enable the LLVM SPIR-V Backend in DPC++ CI. A separate question is how to pass "-fsycl-use-spirv-backend-for-spirv-gen" in the SYCL CTS test suite.

AlexeySachkov

sycl/test-e2e changes LGTM

AlexeySachkov · 2024-12-10T12:36:18Z

.github/workflows/sycl-nightly.yml

I have no objections to that conceptually, but I have a concern that we may not have enough HW available to finish that testing in reasonable time,

Yes, I agree. I will keep the PR draft until we have completed discussing changes and potential consequences.

yeah especially since we're using gen12 which takes forever to run as it is. can you bring this up at our next gatekeeper/CI meeting? there is one today but idk if there is time for this issue. will fwd.

Sure, thank you

sarnex

needs discussion at CI meeting imo

VyacheslavLevytskyy · 2024-12-10T16:17:38Z

needs discussion at CI meeting imo

@sarnex What do you think about the following way forward -- I will remove SYCL CTS from .github/workflows/sycl-nightly.yml, so we have just E2E testing turned on? In this case the blocker for this PR can be addressed separately in the next PR, after we discuss at the CI meeting, and we will be able to proceed without longer pauses.

sarnex · 2024-12-10T16:19:13Z

just E2E in the nightly should be fine IMO, it runs in the middle of the night when nobody is working and CI is empty

but i will ask you to check the results evyery day and make sure its passing and fix/disable failing tests

AlexeySachkov · 2024-12-10T17:09:51Z

sycl/test-e2e/Basic/backend_info.cpp

Ideally, -fsyntax-only check should be moved into check-sycl. That's not a strong comment in this case, because it is a "combined" test (i.e. syntax + execution), but still

sycl/test-e2e/Basic/fpga_tests/fpga_io_pipes.cpp

sycl/test-e2e/NonUniformGroups/is_fixed_topology.cpp

…re 'spirv-backend', substitutions in e2e are aware about the new LIT feature, new records in the sycl-nightly workflow to use the new LIT feature

…unsupported test cases and mark temporarily failing with a known issue with XFAIL/XFAIL-TRACKER; fix how the 'build' substitution is updated for spirv-backend LIT feature

…oject updates

aelovikov-intel · 2025-01-10T17:15:44Z

.github/workflows/sycl-linux-precommit.yml

+            image: ghcr.io/intel/llvm/ubuntu2404_intel_drivers:latest
+            image_options: -u 1001 --device=/dev/dri --privileged --cap-add SYS_ADMIN
+            target_devices: opencl:cpu
+            extra_lit_opts: -j 50 --param spirv-backend=True


This makes no sense, these runners have desktop CPUs and you'll oversubscribe them.

aelovikov-intel · 2025-01-10T17:28:42Z

.github/workflows/sycl-nightly.yml

+            image_options: -u 1001 --privileged --cap-add SYS_ADMIN
+            target_devices: opencl:cpu
+            tests_selector: e2e
+            extra_lit_opts: --param spirv-backend=True --param gpu-intel-gen12=True


gpu-intel-gen12 isn't required after #16220.

aelovikov-intel

I think this should be split into several patches:

sycl-linux-run-tests.yml so that the testing can be dispatched manually.
Update tests and verify via manual CI runs
CI enabling in pre-commit/nightly. Shouldn't include many of the tests changes at that poing.

I also see some supposedly functional changes, they should go into separate PR as well.

YuriPlyakhin

Joint Matrix change LGTM

dkhaldi

JM changes LGTM

VyacheslavLevytskyy · 2025-01-13T11:01:29Z

It was agreed that we split #16317 into three parts:

CI
Clang driver
XFAIL & UNSUPPORTED in end-to-end tests.

I reverted changes in CI/Clang here, so that this PR is only about introducing XFAIL & UNSUPPORTED into end-to-end tests now.

@sarnex Can you please have a look and unblock if everything is ok

aelovikov-intel · 2025-01-13T15:55:40Z

@sarnex Can you please have a look and unblock if everything is ok

IMO, this PR shouldn't even be outside draft state until #16600 is merged in.

joeatodd

syclcompat changes LGTM

VyacheslavLevytskyy · 2025-01-16T12:54:46Z

@sarnex @aelovikov-intel We have #16600 and #16602 merged. This PR contains only changes in test cases, and we got them approved as well. Are we good to proceed with a merge of this PR?

sarnex

lgtm just typos

sycl/test-e2e/InvokeSimd/lit.local.cfg

sycl/test-e2e/ESIMD/lit.local.cfg

aelovikov-intel · 2025-01-16T16:41:43Z

@sarnex @aelovikov-intel We have #16600 and #16602 merged. This PR contains only changes in test cases, and we got them approved as well. Are we good to proceed with a merge of this PR?

I'd like to see some CI results before doing that. Are we in a state when that can be done with manual dispatch of "SYCL E2E" task?

Co-authored-by: Nick Sarnie <[email protected]>

VyacheslavLevytskyy · 2025-01-16T17:20:26Z

@sarnex @aelovikov-intel We have #16600 and #16602 merged. This PR contains only changes in test cases, and we got them approved as well. Are we good to proceed with a merge of this PR?

I'd like to see some CI results before doing that. Are we in a state when that can be done with manual dispatch of "SYCL E2E" task?

Thank you for looking into it. However, I disagree that this is a prerequisite to merge this PR.

There is no need to run CI at this time. We have results of workflows runs (sycl-nightly.yml and sycl-linux-precommit.yml) from Jan 10, where all was green and all changes in end-to-end tests were verified. There is no sense in races between my updates to e2e tests and routine regular changes in intel/llvm. You will not get actual results until you will be ready to run new (not yet created) CI actions, and you already have fresh enough results from last Friday.
We should postpone any additional runs & checks if we have enough XFAIL & UNSUPPORTED for SPIR-v Backend to stay green until after the moment when CI itself is ready to be updated. This PR doesn't change any yml specifications, as we explicitly discussed and agreed upon, and the whole sense of removing from the initial PR changes in .github/workflows/sycl-nightly.yml & .github/workflows/sycl-linux-precommit.yml was to avoid any prerequisites of such kind for this PR.
It's not the last time when those e2e tests are being updated due to progress in development of SPIR-V Backend, drivers update and pulldowns. This is a continuous activity, and, being a process rather than an event, it can't be a blocker to merge this PR.

I'd propose to merge this PR without any additional verifications. Results from January 10 works as well until we get Github Actions updated and ready to run.

Meanwhile, I appreciate your suggestion to use actions/workflows/sycl-linux-run-tests.yml, and without any doubts will use this tool to establish a proper process of syncing SPIR-V Backend progress with SYCL end-to-end test suite.

aelovikov-intel

Before this PR: https://github.com/intel/llvm/actions/runs/12814470669 (581 fails)
With this PR: https://github.com/intel/llvm/actions/runs/12815598260 (4 fails)

VyacheslavLevytskyy · 2025-01-16T20:20:02Z

Before this PR: https://github.com/intel/llvm/actions/runs/12814470669 (581 fails) With this PR: https://github.com/intel/llvm/actions/runs/12815598260 (4 fails)

Thank you. I will update those prior to CI is updated.

AlexeySachkov approved these changes Dec 10, 2024

View reviewed changes

sarnex requested changes Dec 10, 2024

View reviewed changes

AlexeySachkov reviewed Dec 10, 2024

View reviewed changes

VyacheslavLevytskyy force-pushed the introduce_SPIRV_Backend branch 2 times, most recently from 36072bb to c75b88d Compare January 8, 2025 14:17

VyacheslavLevytskyy had a problem deploying to WindowsCILock January 8, 2025 14:55 — with GitHub Actions Failure

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 8, 2025 15:44 — with GitHub Actions Inactive

VyacheslavLevytskyy had a problem deploying to WindowsCILock January 8, 2025 16:41 — with GitHub Actions Failure

VyacheslavLevytskyy had a problem deploying to WindowsCILock January 8, 2025 17:33 — with GitHub Actions Error

VyacheslavLevytskyy had a problem deploying to WindowsCILock January 8, 2025 18:02 — with GitHub Actions Failure

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 8, 2025 18:41 — with GitHub Actions Inactive

VyacheslavLevytskyy had a problem deploying to WindowsCILock January 9, 2025 09:28 — with GitHub Actions Failure

VyacheslavLevytskyy had a problem deploying to WindowsCILock January 9, 2025 10:04 — with GitHub Actions Failure

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 9, 2025 10:54 — with GitHub Actions Inactive

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 9, 2025 11:35 — with GitHub Actions Inactive

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 9, 2025 12:36 — with GitHub Actions Inactive

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 9, 2025 13:16 — with GitHub Actions Inactive

VyacheslavLevytskyy added 10 commits January 9, 2025 09:17

introduce SPIR-V Backend support in LIT tests and CI: a new LIT featu…

c1a7ad6

…re 'spirv-backend', substitutions in e2e are aware about the new LIT feature, new records in the sycl-nightly workflow to use the new LIT feature

fix --param value

0644c4d

Update a command to execute SPIRV-V Backend in the toolchain; Update …

8a54a5d

…unsupported test cases and mark temporarily failing with a known issue with XFAIL/XFAIL-TRACKER; fix how the 'build' substitution is updated for spirv-backend LIT feature

Postpone the question of running SYCL CTS to a next PR

213bcdd

move 'build-only' tests into sycl/test

275ad48

Update test cases with XFAIL/UNSUPPORTED after intel/llvm and llvm-pr…

ad767a0

…oject updates

clang-format

f3a2bad

%{build} -> %clangxx in sycl/test/

c52b5ca

Fix RUN lines in tests, move FPGA header to tests

5bd7b82

Fix clang test for using SPIR-V backend for SYCL offloading

f3ec0ec

aelovikov-intel reviewed Jan 10, 2025

View reviewed changes

YuriPlyakhin approved these changes Jan 10, 2025

View reviewed changes

dkhaldi approved these changes Jan 11, 2025

View reviewed changes

VyacheslavLevytskyy changed the title ~~[E2E][CI][CTS] Introduce SPIR-V Backend support in LIT tests and CI~~ [E2E] Introduce SPIR-V Backend support in SYCL tests and end-to-end tests Jan 13, 2025

This was referenced Jan 13, 2025

[CI] Introduce SPIR-V Backend support in LIT config #16600

Merged

[Clang] Update of invoking SPIR-V Backend in Clang/Driver #16602

Merged

revert chanches in CI and Clang/Driver

92e3ac7

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 13, 2025 10:57 — with GitHub Actions Inactive

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 13, 2025 12:14 — with GitHub Actions Inactive

AllanZyne approved these changes Jan 14, 2025

View reviewed changes

joeatodd approved these changes Jan 14, 2025

View reviewed changes

Merge branch 'sycl' into introduce_SPIRV_Backend

d174300

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 16, 2025 11:05 — with GitHub Actions Inactive

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 16, 2025 11:36 — with GitHub Actions Inactive

sarnex approved these changes Jan 16, 2025

View reviewed changes

sycl/test-e2e/InvokeSimd/lit.local.cfg Outdated Show resolved Hide resolved

sycl/test-e2e/ESIMD/lit.local.cfg Outdated Show resolved Hide resolved

VyacheslavLevytskyy and others added 2 commits January 16, 2025 17:55

Update sycl/test-e2e/InvokeSimd/lit.local.cfg

45ac954

Co-authored-by: Nick Sarnie <[email protected]>

Update sycl/test-e2e/ESIMD/lit.local.cfg

d23d5eb

Co-authored-by: Nick Sarnie <[email protected]>

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 16, 2025 16:57 — with GitHub Actions Inactive

MrSidims approved these changes Jan 16, 2025

View reviewed changes

VyacheslavLevytskyy temporarily deployed to WindowsCILock January 16, 2025 17:48 — with GitHub Actions Inactive

aelovikov-intel approved these changes Jan 16, 2025

View reviewed changes

aelovikov-intel merged commit 1bd15f0 into intel:sycl Jan 16, 2025
17 checks passed

[E2E] Introduce SPIR-V Backend support in SYCL tests and end-to-end tests #16317

[E2E] Introduce SPIR-V Backend support in SYCL tests and end-to-end tests #16317

Uh oh!

Conversation

VyacheslavLevytskyy commented Dec 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

VyacheslavLevytskyy commented Dec 10, 2024

Uh oh!

AlexeySachkov left a comment

Choose a reason for hiding this comment

Uh oh!

AlexeySachkov Dec 10, 2024

Choose a reason for hiding this comment

Uh oh!

VyacheslavLevytskyy Dec 10, 2024

Choose a reason for hiding this comment

Uh oh!

sarnex Dec 10, 2024

Choose a reason for hiding this comment

Uh oh!

VyacheslavLevytskyy Dec 10, 2024

Choose a reason for hiding this comment

Uh oh!

sarnex left a comment

Choose a reason for hiding this comment

Uh oh!

VyacheslavLevytskyy commented Dec 10, 2024

Uh oh!

sarnex commented Dec 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexeySachkov Dec 10, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

aelovikov-intel Jan 10, 2025

Choose a reason for hiding this comment

Uh oh!

aelovikov-intel Jan 10, 2025

Choose a reason for hiding this comment

Uh oh!

aelovikov-intel left a comment

Choose a reason for hiding this comment

Uh oh!

YuriPlyakhin left a comment

Choose a reason for hiding this comment

Uh oh!

dkhaldi left a comment

Choose a reason for hiding this comment

Uh oh!

VyacheslavLevytskyy commented Jan 13, 2025

Uh oh!

aelovikov-intel commented Jan 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joeatodd left a comment

Choose a reason for hiding this comment

Uh oh!

VyacheslavLevytskyy commented Jan 16, 2025

Uh oh!

sarnex left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

aelovikov-intel commented Jan 16, 2025

Uh oh!

VyacheslavLevytskyy commented Jan 16, 2025

Uh oh!

aelovikov-intel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

VyacheslavLevytskyy commented Jan 16, 2025

Uh oh!

Reviewers

Assignees

Labels

VyacheslavLevytskyy commented Dec 10, 2024 •

edited

Loading

sarnex commented Dec 10, 2024 •

edited

Loading

aelovikov-intel commented Jan 13, 2025 •

edited

Loading