[SYCL] Fix SYCL_DEVICE_FILTER to affect get_devices()/get_platforms() (#3397)

SYCL_DEVICE_FILTER will now throw an exception if no matched device is found
regardless of the device_selector type.
Host device is not automatically available if SYCL_DEVICE_FILTER is set.
Tests configurations and documentation updated.

Signed-off-by: Byoungro So <[email protected]>

Author: bso-intel

sycl/doc/EnvironmentVariables.md

@@ -12,9 +12,9 @@ subject to change. Do not rely on these variables in production code.
 | Environment variable | Values | Description |
 | -------------------- | ------ | ----------- |
 | SYCL_PI_TRACE | Described [below](#sycl_pi_trace-options)  | Enable specified level of tracing for PI. |
-| SYCL_BE | PI_OPENCL, PI_LEVEL_ZERO, PI_CUDA | Force SYCL RT to consider only devices of the specified backend during the device selection. We are planning to deprecate SYCL_BE environment variable in the future. The specific grace period is not decided yet. Please use the new env var SYCL_DEVICE_FILTER instead. |
-| SYCL_DEVICE_TYPE | One of: CPU, GPU, ACC, HOST | Force SYCL to use the specified device type. If unset, default selection rules are applied. If set to any unlisted value, this control has no effect. If the requested device type is not found, a `cl::sycl::runtime_error` exception is thrown. If a non-default device selector is used, a device must satisfy both the selector and this control to be chosen. This control only has effect on devices created with a selector. We are planning to deprecate SYCL_DEVICE_TYPE environment variable in the future. The specific grace period is not decided yet. Please use the new env var SYCL_DEVICE_FILTER instead. |
-| SYCL_DEVICE_FILTER (tentative name) | {backend:device_type:device_num} | Limits the SYCL RT to use only a subset of the system's devices. Setting this environment variable affects all of the device query functions and all of the device selectors. The value of this environment variable is a comma separated list of filters, where each filter is a triple of the form "backend:device_type:device_num" (without the quotes). Each element of the triple is optional, but each filter must have at least one value. Possible values of "backend" are "host", "level_zero", "opencl", "cuda", or "\*". Possible values of "device_type" are "host", "cpu", "gpu", "acc", or "\*". Device_num is an integer that indexes the enumeration of devices from the sycl-ls utility tool, where the first device in that enumeration has index zero. Assuming a filter has all three elements of the triple, it selects only those devices that come from the given backend, have the specified device type, AND have the given device index. If more than one filter is specified, the RT is restricted to the union of devices selected by all filters. The RT always includes the "host" backend and the host device regardless of the filter because the SYCL language requires this device to always be present. Therefore, one can specify 'host' to enforce SYCL to use the host device. Note that the standard selectors like gpu_selector or cpu_selector will throw an exception if the filtered list of devices does not include a device that satisfies the selector. In particular, limiting the devices to only those supported by the "level_zero" backend will cause the cpu_selector to throw an exception since that backend does not support any CPU devices at this time. This environment variable can be used to limit loading only specified plugins into the SYCL RT. |
+| SYCL_BE (deprecated) | PI_OPENCL, PI_LEVEL_ZERO, PI_CUDA | Force SYCL RT to consider only devices of the specified backend during the device selection. We are planning to deprecate SYCL_BE environment variable in the future. The specific grace period is not decided yet. Please use the new env var SYCL_DEVICE_FILTER instead. |
+| SYCL_DEVICE_TYPE (deprecated) | CPU, GPU, ACC, HOST | Force SYCL to use the specified device type. If unset, default selection rules are applied. If set to any unlisted value, this control has no effect. If the requested device type is not found, a `cl::sycl::runtime_error` exception is thrown. If a non-default device selector is used, a device must satisfy both the selector and this control to be chosen. This control only has effect on devices created with a selector. We are planning to deprecate SYCL_DEVICE_TYPE environment variable in the future. The specific grace period is not decided yet. Please use the new env var SYCL_DEVICE_FILTER instead. |
+| SYCL_DEVICE_FILTER | backend:device_type:device_num | See Section [SYCL_DEVICE_FILTER](#sycl_device_filter)  below. |
 | SYCL_PROGRAM_COMPILE_OPTIONS | String of valid OpenCL compile options | Override compile options for all programs. |
 | SYCL_PROGRAM_LINK_OPTIONS | String of valid OpenCL link options | Override link options for all programs. |
 | SYCL_USE_KERNEL_SPV | Path to the SPIR-V binary | Load device image from the specified file. If runtime is unable to read the file, `cl::sycl::runtime_error` exception is thrown.|
@@ -46,6 +46,30 @@ subject to change. Do not rely on these variables in production code.
 
 `(*) Note: Any means this environment variable is effective when set to any non-null value.`
 
+### SYCL_DEVICE_FILTER
+
+This environment variable limits the SYCL RT to use only a subset of the system's devices. Setting this environment variable affects all of the device query functions (platform::get_devices() and platform::get_platforms()) and all of the device selectors. 
+
+The value of this environment variable is a comma separated list of filters, where each filter is a triple of the form "backend:device_type:device_num" (without the quotes). Each element of the triple is optional, but each filter must have at least one value. Possible values of "backend" are:
+- host
+- level_zero
+- opencl
+- cuda
+- \*
+
+Possible values of "device_type" are:
+- host
+- cpu
+- gpu
+- acc
+- \*
+
+Device_num is an integer that indexes the enumeration of devices from the sycl-ls utility tool, where the first device in that enumeration has index zero in each backend. For example, SYCL_DEVICE_FILTER=2 will return all devices with index '2' from all different backends. If multiple devices satisfy this device number (e.g., GPU and CPU devices can be assigned device number '2'), then default_selector will choose the device with the highest heuristic point. 
+
+Assuming a filter has all three elements of the triple, it selects only those devices that come from the given backend, have the specified device type, AND have the given device index. If more than one filter is specified, the RT is restricted to the union of devices selected by all filters. The RT does not include the "host" backend and the host device automatically unless one of the filters explicitly specifies the "host" device type. Therefore, SYCL_DEVICE_FILTER=host should be set to enforce SYCL to use the host device only. 
+
+Note that all device selectors will throw an exception if the filtered list of devices does not include a device that satisfies the selector. For instance, SYCL_DEVICE_FILTER=cpu,level_zero will cause host_selector() to throw an exception. SYCL_DEVICE_FILTER also limits loading only specified plugins into the SYCL RT. In particular, SYCL_DEVICE_FILTER=level_zero will cause the cpu_selector to throw an exception since SYCL RT will only load the level_zero backend which does not support any CPU devices at this time. When multiple devices satisfy the filter (e..g, SYCL_DEVICE_FILTER=gpu), only one of them will be selected.
+
 ### SYCL_PRINT_EXECUTION_GRAPH Options
 
 SYCL_PRINT_EXECUTION_GRAPH can accept one or more comma separated values from the table below

sycl/include/CL/sycl/detail/device_filter.hpp

@@ -43,6 +43,10 @@ class device_filter_list {
   device_filter_list(device_filter &Filter);
   void addFilter(device_filter &Filter);
   std::vector<device_filter> &get() { return FilterList; }
+  bool backendCompatible(backend Backend);
+  bool deviceTypeCompatible(info::device_type DeviceType);
+  bool deviceNumberCompatible(int DeviceNum);
+  bool containsHost();
   friend std::ostream &operator<<(std::ostream &Out,
                                   const device_filter_list &List);
 };

sycl/source/detail/device_filter.cpp

@@ -124,6 +124,49 @@ void device_filter_list::addFilter(device_filter &Filter) {
   FilterList.push_back(Filter);
 }
 
+// Backend is compatible with the SYCL_DEVICE_FILTER in the following cases.
+// 1. Filter backend is '*' which means ANY backend.
+// 2. Filter backend match exactly with the given 'Backend'
+bool device_filter_list::backendCompatible(backend Backend) {
+  for (const device_filter &Filter : FilterList) {
+    backend FilterBackend = Filter.Backend;
+    if (FilterBackend == Backend || FilterBackend == backend::all)
+      return true;
+  }
+  return false;
+}
+
+bool device_filter_list::deviceTypeCompatible(info::device_type DeviceType) {
+  for (const device_filter &Filter : FilterList) {
+    info::device_type FilterDevType = Filter.DeviceType;
+    if (FilterDevType == DeviceType || FilterDevType == info::device_type::all)
+      return true;
+  }
+  return false;
+}
+
+bool device_filter_list::deviceNumberCompatible(int DeviceNum) {
+  for (const device_filter &Filter : FilterList) {
+    int FilterDevNum = Filter.DeviceNum;
+    if (!Filter.HasDeviceNum || FilterDevNum == DeviceNum)
+      return true;
+  }
+  return false;
+}
+
+bool device_filter_list::containsHost() {
+  for (const device_filter &Filter : FilterList) {
+    if (Filter.Backend == backend::host || Filter.Backend == backend::all)
+      if (Filter.DeviceType == info::device_type::host ||
+          Filter.DeviceType == info::device_type::all)
+        // SYCL RT never creates more than one HOST device.
+        // All device numbers other than 0 are rejected.
+        if (!Filter.HasDeviceNum || Filter.DeviceNum == 0)
+          return true;
+  }
+  return false;
+}
+
 } // namespace detail
 } // namespace sycl
 } // __SYCL_INLINE_NAMESPACE(cl)

sycl/source/detail/platform_impl.cpp

@@ -123,8 +123,12 @@ vector_class<platform> platform_impl::get_platforms() {
     }
   }
 
-  // The host platform should always be available.
-  Platforms.emplace_back(platform());
+  // The host platform should always be available unless not allowed by the
+  // SYCL_DEVICE_FILTER
+  detail::device_filter_list *FilterList =
+      detail::SYCLConfig<detail::SYCL_DEVICE_FILTER>::get();
+  if (!FilterList || FilterList->backendCompatible(backend::host))
+    Platforms.emplace_back(platform());
 
   return Platforms;
 }
@@ -287,6 +291,55 @@ static void filterAllowList(vector_class<RT::PiDevice> &PiDevices,
   PiDevices.resize(InsertIDx);
 }
 
+// Filter out the devices that are not compatible with SYCL_DEVICE_FILTER.
+// All three entries (backend:device_type:device_num) are optional.
+// The missing entries are constructed using '*', which means 'any' | 'all'
+// by the device_filter constructor.
+// This function matches devices in the order of backend, device_type, and
+// device_num.
+static void filterDeviceFilter(vector_class<RT::PiDevice> &PiDevices,
+                               const plugin &Plugin) {
+  device_filter_list *FilterList = SYCLConfig<SYCL_DEVICE_FILTER>::get();
+  if (!FilterList)
+    return;
+
+  backend Backend = Plugin.getBackend();
+  int InsertIDx = 0;
+  int DeviceNum = 0;
+  for (RT::PiDevice Device : PiDevices) {
+    RT::PiDeviceType PiDevType;
+    Plugin.call<PiApiKind::piDeviceGetInfo>(Device, PI_DEVICE_INFO_TYPE,
+                                            sizeof(RT::PiDeviceType),
+                                            &PiDevType, nullptr);
+    // Assumption here is that there is 1-to-1 mapping between PiDevType and
+    // Sycl device type for GPU, CPU, and ACC.
+    info::device_type DeviceType = pi::cast<info::device_type>(PiDevType);
+
+    for (const device_filter &Filter : FilterList->get()) {
+      backend FilterBackend = Filter.Backend;
+      // First, match the backend entry
+      if (FilterBackend == Backend || FilterBackend == backend::all) {
+        info::device_type FilterDevType = Filter.DeviceType;
+        // Next, match the device_type entry
+        if (FilterDevType == info::device_type::all) {
+          // Last, match the device_num entry
+          if (!Filter.HasDeviceNum || DeviceNum == Filter.DeviceNum) {
+            PiDevices[InsertIDx++] = Device;
+            break;
+          }
+        } else if (FilterDevType == DeviceType) {
+          if (!Filter.HasDeviceNum || DeviceNum == Filter.DeviceNum) {
+            PiDevices[InsertIDx++] = Device;
+            break;
+          }
+        }
+      }
+    }
+    DeviceNum++;
+  }
+  PiDevices.resize(InsertIDx);
+}
+
 std::shared_ptr<device_impl> platform_impl::getOrMakeDeviceImpl(
     RT::PiDevice PiDevice, const std::shared_ptr<platform_impl> &PlatformImpl) {
   const std::lock_guard<std::mutex> Guard(MDeviceMapMutex);
@@ -312,7 +365,11 @@ platform_impl::get_devices(info::device_type DeviceType) const {
   vector_class<device> Res;
   if (is_host() && (DeviceType == info::device_type::host ||
                     DeviceType == info::device_type::all)) {
-    Res.push_back(device());
+    // If SYCL_DEVICE_FILTER is set, check if filter contains host.
+    device_filter_list *FilterList = SYCLConfig<SYCL_DEVICE_FILTER>::get();
+    if (!FilterList || FilterList->containsHost()) {
+      Res.push_back(device());
+    }
   }
 
   // If any DeviceType other than host was requested for host platform,
@@ -339,6 +396,9 @@ platform_impl::get_devices(info::device_type DeviceType) const {
   if (SYCLConfig<SYCL_DEVICE_ALLOWLIST>::get())
     filterAllowList(PiDevices, MPlatform, this->getPlugin());
 
+  // Filter out devices that are not compatible with SYCL_DEVICE_FILTER
+  filterDeviceFilter(PiDevices, Plugin);
+
   PlatformImplPtr PlatformImpl = getOrMakePlatformImpl(MPlatform, *MPlugin);
   std::transform(
       PiDevices.begin(), PiDevices.end(), std::back_inserter(Res),

sycl/source/device.cpp

@@ -6,6 +6,7 @@
 //
 //===----------------------------------------------------------------------===//
 
+#include <CL/sycl/detail/device_filter.hpp>
 #include <CL/sycl/detail/export.hpp>
 #include <CL/sycl/device.hpp>
 #include <CL/sycl/device_selector.hpp>
@@ -49,9 +50,21 @@ device::device(const device_selector &deviceSelector) {
 
 vector_class<device> device::get_devices(info::device_type deviceType) {
   vector_class<device> devices;
-  // Host device availability should not depend on the forced type
-  const bool includeHost =
-      detail::match_types(deviceType, info::device_type::host);
+  detail::device_filter_list *FilterList =
+      detail::SYCLConfig<detail::SYCL_DEVICE_FILTER>::get();
+  // Host device availability should depend on the forced type
+  bool includeHost = false;
+  // If SYCL_DEVICE_FILTER is set, we don't automatically include it.
+  // We will check if host devices are specified in the filter below.
+  if (FilterList) {
+    if (deviceType != info::device_type::host &&
+        deviceType != info::device_type::all)
+      includeHost = false;
+    else
+      includeHost = FilterList->containsHost();
+  } else {
+    includeHost = detail::match_types(deviceType, info::device_type::host);
+  }
   info::device_type forced_type = detail::get_forced_type();
   // Exclude devices which do not match requested device type
   if (detail::match_types(deviceType, forced_type)) {
@@ -61,10 +74,13 @@ vector_class<device> device::get_devices(info::device_type deviceType) {
       // backend.
       backend *ForcedBackend = detail::SYCLConfig<detail::SYCL_BE>::get();
       if (ForcedBackend)
-        if (!plt.is_host() &&
-            (detail::getSyclObjImpl(plt)->getPlugin().getBackend() !=
-             *ForcedBackend))
+        if (!plt.is_host() && plt.get_backend() != *ForcedBackend)
           continue;
+      // If SYCL_DEVICE_FILTER is set, skip platforms that is incompatible
+      // with the filter specification.
+      if (FilterList && !FilterList->backendCompatible(plt.get_backend()))
+        continue;
+
       if (includeHost && plt.is_host()) {
         vector_class<device> host_device(
             plt.get_devices(info::device_type::host));
@@ -78,7 +94,6 @@ vector_class<device> device::get_devices(info::device_type deviceType) {
       }
     }
   }
-
   return devices;
 }
 

sycl/source/device_selector.cpp

@@ -137,13 +137,18 @@ int default_selector::operator()(const device &dev) const {
   if (isDeviceOfPreferredSyclBe(dev))
     Score = 50;
 
-  // override always wins
-  // filter device gets a high point.
-  if (isForcedDevice(dev))
-    Score += 1000;
+  // If SYCL_DEVICE_FILTER is set, filter device gets a high point.
+  // All unmatched devices should never be selected.
+  detail::device_filter_list *FilterList =
+      detail::SYCLConfig<detail::SYCL_DEVICE_FILTER>::get();
+  if (FilterList) {
+    if (isForcedDevice(dev))
+      Score = 1000;
+    else
+      return REJECT_DEVICE_SCORE;
+  }
 
-  else if (dev.get_info<info::device::device_type>() ==
-           detail::get_forced_type())
+  if (dev.get_info<info::device::device_type>() == detail::get_forced_type())
     Score += 1000;
 
   if (dev.is_gpu())

sycl/test/Unit/lit.cfg.py

@@ -73,5 +73,5 @@ def find_shlibpath_var():
     lit_config.warning("unable to inject shared library path on '{}'"
                        .format(platform.system()))
 
-config.environment['SYCL_DEVICE_FILTER'] = lit_config.params.get('SYCL_PLUGIN', "opencl")
+config.environment['SYCL_DEVICE_FILTER'] = lit_config.params.get('SYCL_PLUGIN', "opencl") + ",host"
 lit_config.note("Backend: {}".format(config.environment['SYCL_DEVICE_FILTER']))

sycl/test/on-device/lit.cfg.py

@@ -188,11 +188,11 @@ def getDeviceCount(device_type):
 if getDeviceCount("cpu")[0]:
     found_at_least_one_device = True
     lit_config.note("Found available CPU device")
-    cpu_run_substitute = "env SYCL_DEVICE_FILTER={SYCL_PLUGIN}:cpu ".format(SYCL_PLUGIN=backend)
+    cpu_run_substitute = "env SYCL_DEVICE_FILTER=cpu,host "
     cpu_check_substitute = "| FileCheck %s"
     config.available_features.add('cpu')
     if platform.system() == "Linux":
-        cpu_run_on_linux_substitute = "env SYCL_DEVICE_FILTER={SYCL_PLUGIN}:cpu ".format(SYCL_PLUGIN=backend)
+        cpu_run_on_linux_substitute = "env SYCL_DEVICE_FILTER=cpu,host "
         cpu_check_on_linux_substitute = "| FileCheck %s"
 else:
     lit_config.warning("CPU device not found")
@@ -214,7 +214,7 @@ def getDeviceCount(device_type):
 if gpu_count > 0:
     found_at_least_one_device = True
     lit_config.note("Found available GPU device")
-    gpu_run_substitute = " env SYCL_DEVICE_FILTER={SYCL_PLUGIN}:gpu ".format(SYCL_PLUGIN=backend)
+    gpu_run_substitute = " env SYCL_DEVICE_FILTER={SYCL_PLUGIN}:gpu,host ".format(SYCL_PLUGIN=backend)
     gpu_check_substitute = "| FileCheck %s"
     config.available_features.add('gpu')
     if cuda:
@@ -223,7 +223,7 @@ def getDeviceCount(device_type):
        config.available_features.add('level_zero')
 
     if platform.system() == "Linux":
-        gpu_run_on_linux_substitute = "env SYCL_DEVICE_FILTER={SYCL_PLUGIN}:gpu ".format(SYCL_PLUGIN=backend)
+        gpu_run_on_linux_substitute = "env SYCL_DEVICE_FILTER={SYCL_PLUGIN}:gpu,host ".format(SYCL_PLUGIN=backend)
         gpu_check_on_linux_substitute = "| FileCheck %s"
 else:
     lit_config.warning("GPU device not found")
@@ -238,7 +238,7 @@ def getDeviceCount(device_type):
 if getDeviceCount("accelerator")[0]:
     found_at_least_one_device = True
     lit_config.note("Found available accelerator device")
-    acc_run_substitute = " env SYCL_DEVICE_FILTER={SYCL_PLUGIN}:acc ".format(SYCL_PLUGIN=backend)
+    acc_run_substitute = " env SYCL_DEVICE_FILTER=acc "
     acc_check_substitute = "| FileCheck %s"
     config.available_features.add('accelerator')
 else:

sycl/tools/sycl-ls/sycl-ls.cpp

@@ -111,11 +111,9 @@ int main(int argc, char **argv) {
     std::cout << "Platforms: " << Platforms.size() << std::endl;
 
   uint32_t PlatformNum = 0;
-  // DeviceNum represents a globally unique device number.
-  // It is printed at the beginning of each line from 'sycl-ls'.
-  // This number starts at 0.
-  uint32_t DeviceNum = 0;
+
   for (const auto &Platform : Platforms) {
+    uint32_t DeviceNum = 0;
     ++PlatformNum;
     if (verbose) {
       auto PlatformVersion = Platform.get_info<info::platform::version>();
@@ -132,8 +130,10 @@ int main(int argc, char **argv) {
     for (const auto &Device : Devices) {
       if (verbose)
         std::cout << "        Device [#" << DeviceNum << "]:" << std::endl;
-      else
-        std::cout << DeviceNum << ". ";
+      else {
+        backend Backend = Platform.get_backend();
+        std::cout << "[" << Backend << ":" << DeviceNum << "] ";
+      }
       ++DeviceNum;
       printDeviceInfo(Device, verbose ? "        " : "");
     }