[SYCL] Change the definition of device number to use the 'sycl-ls' tool (#3095)

* [SYCL] Sync the device enumeration methods in sycl-ls and SYCL_DEVICE_FILTER

In recent discussion, we decided to change the definition of device number in SYCL_DEVICE_FILTER.
Originally, it was the enumeration order from get_devices() API.
But, it is better to use the sycl-ls enumeration order, which actually uses the same device enumeration order.

Signed-off-by: Byoungro So <[email protected]>

Author: bso-intel

sycl/doc/EnvironmentVariables.md

@@ -14,7 +14,7 @@ subject to change. Do not rely on these variables in production code.
 | SYCL_PI_TRACE | Described [below](#sycl_pi_trace-options)  | Enable specified level of tracing for PI. |
 | SYCL_BE | PI_OPENCL, PI_LEVEL_ZERO, PI_CUDA | Force SYCL RT to consider only devices of the specified backend during the device selection. We are planning to deprecate SYCL_BE environment variable in the future. The specific grace period is not decided yet. Please use the new env var SYCL_DEVICE_FILTER instead. |
 | SYCL_DEVICE_TYPE | One of: CPU, GPU, ACC, HOST | Force SYCL to use the specified device type. If unset, default selection rules are applied. If set to any unlisted value, this control has no effect. If the requested device type is not found, a `cl::sycl::runtime_error` exception is thrown. If a non-default device selector is used, a device must satisfy both the selector and this control to be chosen. This control only has effect on devices created with a selector. We are planning to deprecate SYCL_DEVICE_TYPE environment variable in the future. The specific grace period is not decided yet. Please use the new env var SYCL_DEVICE_FILTER instead. |
-| SYCL_DEVICE_FILTER (tentative name) | {backend:device_type:device_num} | Limits the SYCL RT to use only a subset of the system's devices. Setting this environment variable affects all of the device query functions and all of the device selectors. The value of this environment variable is a comma separated list of filters, where each filter is a triple of the form "backend:device_type:device_num" (without the quotes). Each element of the triple is optional, but each filter must have at least one value. Possible values of "backend" are "host", "level_zero", "opencl", "cuda", or "\*". Possible values of "device_type" are "host", "cpu", "gpu", "acc", or "\*". Device_num is an integer that indexes the enumeration of devices from the sycl::platform::get_device() call, where the first device in that enumeration has index zero. Assuming a filter has all three elements of the triple, it selects only those devices that come from the given backend, have the specified device type, AND have the given device index. If more than one filter is specified, the RT is restricted to the union of devices selected by all filters. The RT always includes the "host" backend and the host device regardless of the filter because the SYCL language requires this device to always be present. Therefore, including "host" in the list of filters is allowed but is unnecessary. Note that the standard selectors like gpu_selector or cpu_selector will throw an exception if the filtered list of devices does not include a device that satisfies the selector. In particular, limiting the devices to only those supported by the "level_zero" backend will cause the cpu_selector to throw an exception since that backend does not support any CPU devices. This environment variable can be used to limit loading only specified plugins into the SYCL RT. |
+| SYCL_DEVICE_FILTER (tentative name) | {backend:device_type:device_num} | Limits the SYCL RT to use only a subset of the system's devices. Setting this environment variable affects all of the device query functions and all of the device selectors. The value of this environment variable is a comma separated list of filters, where each filter is a triple of the form "backend:device_type:device_num" (without the quotes). Each element of the triple is optional, but each filter must have at least one value. Possible values of "backend" are "host", "level_zero", "opencl", "cuda", or "\*". Possible values of "device_type" are "host", "cpu", "gpu", "acc", or "\*". Device_num is an integer that indexes the enumeration of devices from the sycl-ls utility tool, where the first device in that enumeration has index zero. Assuming a filter has all three elements of the triple, it selects only those devices that come from the given backend, have the specified device type, AND have the given device index. If more than one filter is specified, the RT is restricted to the union of devices selected by all filters. The RT always includes the "host" backend and the host device regardless of the filter because the SYCL language requires this device to always be present. Therefore, one can specify 'host' to enforce SYCL to use the host device. Note that the standard selectors like gpu_selector or cpu_selector will throw an exception if the filtered list of devices does not include a device that satisfies the selector. In particular, limiting the devices to only those supported by the "level_zero" backend will cause the cpu_selector to throw an exception since that backend does not support any CPU devices at this time. This environment variable can be used to limit loading only specified plugins into the SYCL RT. |
 | SYCL_PROGRAM_COMPILE_OPTIONS | String of valid OpenCL compile options | Override compile options for all programs. |
 | SYCL_PROGRAM_LINK_OPTIONS | String of valid OpenCL link options | Override link options for all programs. |
 | SYCL_USE_KERNEL_SPV | Path to the SPIR-V binary | Load device image from the specified file. If runtime is unable to read the file, `cl::sycl::runtime_error` exception is thrown.|

sycl/test/regression/device_num.cpp

@@ -0,0 +1,93 @@
+// RUN: %clangxx -fsycl -fsycl-targets=%sycl_triple %s -o %t.out
+// RUN: sycl-ls
+// RUN: env SYCL_DEVICE_FILTER=0 %t.out
+// RUN: env SYCL_DEVICE_FILTER=1 %t.out
+// RUN: env SYCL_DEVICE_FILTER=2 %t.out
+// RUN: env SYCL_DEVICE_FILTER=3 %t.out
+// RUN: env SYCL_DEVICE_FILTER=4 %t.out
+
+#include <CL/sycl.hpp>
+#include <iostream>
+
+using namespace cl::sycl;
+using namespace std;
+
+void printDeviceType(const device &d) {
+  string name = d.get_platform().get_info<info::platform::name>();
+  auto DeviceType = d.get_info<info::device::device_type>();
+  std::string DeviceTypeName;
+
+  switch (DeviceType) {
+  case info::device_type::cpu:
+    DeviceTypeName = "CPU ";
+    break;
+  case info::device_type::gpu:
+    DeviceTypeName = "GPU ";
+    break;
+  case info::device_type::host:
+    DeviceTypeName = "HOST ";
+    break;
+  case info::device_type::accelerator:
+    DeviceTypeName = "ACCELERATOR ";
+    break;
+  default:
+    DeviceTypeName = "UNKNOWN ";
+    break;
+  }
+  std::cout << DeviceTypeName << name << std::endl;
+}
+
+int main() {
+  const char *envVal = std::getenv("SYCL_DEVICE_FILTER");
+  int deviceNum;
+  std::cout << "SYCL_DEVICE_FILTER=" << envVal << std::endl;
+  deviceNum = std::atoi(envVal);
+
+  auto devices = device::get_devices();
+  if (devices.size() >= deviceNum) {
+    device targetDevice = devices[deviceNum];
+    std::cout << "Target Device: ";
+    printDeviceType(targetDevice);
+
+    {
+      default_selector ds;
+      device d = ds.select_device();
+      std::cout << "default_selector selected ";
+      printDeviceType(d);
+      assert(targetDevice == d &&
+             "The selected device is not the target device specified.");
+    }
+
+    if (targetDevice.is_gpu()) {
+      gpu_selector gs;
+      device d = gs.select_device();
+      std::cout << "gpu_selector selected ";
+      printDeviceType(d);
+      assert(targetDevice == d &&
+             "The selected device is not the target device specified.");
+    } else if (targetDevice.is_cpu()) {
+      cpu_selector cs;
+      device d = cs.select_device();
+      std::cout << "cpu_selector selected ";
+      printDeviceType(d);
+      assert(targetDevice == d &&
+             "The selected device is not the target device specified.");
+    } else if (targetDevice.is_accelerator()) {
+      accelerator_selector as;
+      device d = as.select_device();
+      std::cout << "accelerator_selector selected ";
+      printDeviceType(d);
+      assert(targetDevice == d &&
+             "The selected device is not the target device specified.");
+    }
+    // HOST device is always available regardless of SYCL_DEVICE_FILTER
+    {
+      host_selector hs;
+      device d = hs.select_device();
+      std::cout << "host_selector selected ";
+      printDeviceType(d);
+      assert(d.is_host() && "The selected device is not a host device.");
+    }
+  }
+  return 0;
+}

sycl/tools/sycl-ls/sycl-ls.cpp

@@ -111,6 +111,10 @@ int main(int argc, char **argv) {
     std::cout << "Platforms: " << Platforms.size() << std::endl;
 
   uint32_t PlatformNum = 0;
+  // DeviceNum represents a globally unique device number.
+  // It is printed at the beginning of each line from 'sycl-ls'.
+  // This number starts at 0.
+  uint32_t DeviceNum = 0;
   for (const auto &Platform : Platforms) {
     ++PlatformNum;
     if (verbose) {
@@ -125,11 +129,12 @@ int main(int argc, char **argv) {
     const auto &Devices = Platform.get_devices();
     if (verbose)
       std::cout << "    Devices  : " << Devices.size() << std::endl;
-    uint32_t DeviceNum = 0;
     for (const auto &Device : Devices) {
-      ++DeviceNum;
       if (verbose)
         std::cout << "        Device [#" << DeviceNum << "]:" << std::endl;
+      else
+        std::cout << DeviceNum << ". ";
+      ++DeviceNum;
       printDeviceInfo(Device, verbose ? "        " : "");
     }
   }