[SYCL] map cudaMemGetInfo

[zjin-lcf]

I am not sure whether a query of free_byte is supported by SYCL. Thanks.

cudaMemGetInfo(&free_byte, &total_byte);

[romanovvlad]

It seems we have something similar for level zero https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/supported/sycl_ext_oneapi_backend_level_zero.md#51-device-information-descriptors .

[zjin-lcf]

Thank you for the link. Could this be included in the SYCL specification rather than in the level zero ?

[Michoumichmich]

Thank you for the link. Could this be included in the SYCL specification rather than in the level zero ?

I faced the same issue some time ago and it was the idea behind KhronosGroup/SYCL-Docs#207

[zjin-lcf]

It may be true that few CUDA programs call the function. I found one in the CUDA samples.

6_Advanced/segmentationTreeThrust/segmentationTree.cu

[abagusetty]

@zjin-lcf I instead hacked in a bit of L0 inter-op into SYCL and accomplished the same, hope this helps. Some of the functional
This would also need one to set ZES_ENABLE_SYSMAN=1 ans some cleanup with device selection (syclGetDevice, sycl_get_device, etc)

#include "level_zero/ze_api.h"
#include "level_zero/zes_api.h"
#include "CL/sycl/backend/level_zero.hpp"

static void syclMemGetInfo(size_t* m_free, size_t* m_total)
{
  size_t mfree=0, mtotal=0;
  int current_devID=-1;
  syclGetDevice(&current_devID);
  assert(current_devID == -1);
  sycl::device* myDev = sycl_get_device(current_devID);

  ze_device_handle_t ze_device = sycl::get_native<sycl::backend::ext_oneapi_level_zero>(*myDev);
  zes_device_handle_t zes_device = (zes_device_handle_t)ze_device;

  ze_result_t status = ZE_RESULT_SUCCESS;

  uint32_t module_count = 0;
  status = zesDeviceEnumMemoryModules(zes_device, &module_count, nullptr);
  if( module_count > 0 ) {
    std::vector<zes_mem_handle_t> module_list(module_count);
    std::vector<zes_mem_state_t> state_list(module_count);

    status = zesDeviceEnumMemoryModules(zes_device, &module_count, module_list.data());

    for (uint32_t i = 0; i < module_count; ++i) {
      status = zesMemoryGetState(module_list[i], &(state_list[i]));
      mfree += state_list[i].free;
      mtotal += state_list[i].size;
    }
  }

  *m_free  = mfree;
  *m_total = mtotal;
  return;
}

[zjin-lcf]

@keryell, could this be included in the SYCL specification ?

info::device::memory_info,
info::device::free_memory,
info::device::total_memory,

@abagusetty Thank you for writing a SYCL function for cudaMemGetInfo.

[abagusetty]

@zjin-lcf Just checking on this, if you had a chance to explore the following recently added extension to your issue and update this accordingly.

#include <sycl/sycl.hpp>

int main() {
  auto const& gpu_devices = sycl::device::get_devices(sycl::info::device_type::gpu);
  std::cout << "Number of Root GPUs: " << gpu_devices.size() << std::endl;

  for(const auto& d : gpu_devices) {
    std::cout << "(root-dev) GPU-ID: " << d.get_info<sycl::info::device::name>() << std::endl;
    size_t totalMemory = d.get_info<sycl::info::device::global_mem_size>();
    size_t freeMemory = d.get_info<sycl::ext::intel::info::device::free_memory>();
    std::cout << "(root-dev) TotalMem (bytes): " << totalMemory << ", FreeMem (bytes): " << freeMemory << std::endl;
  }

  return 0;
}

[zjin-lcf]

@abagusetty Thank you for the update. Could sycl::ext::intel::info:: be renamed to sycl::info:: ?

[abagusetty]

I believe this is still inside an extension that is yet to be ratified the SYCL specs. Hence the ::ext::intel::info

[zjin-lcf]

@abagusetty
Could you reproduce the errors using the program you listed ?

Number of Root GPUs: 4
(root-dev) GPU-ID: Tesla V100-DGXS-32GB
warning: Cuda API error detected: cuMemGetInfo_v2 returned (0xc9)

pi_die: failed cuMemGetInfo() API.
terminate called without an active exception

[abagusetty]

@zjin-lcf Thanks for pointing it out. I too see this error from the above test code on A100, 11.8 CUDA SDK

Number of Root GPUs: 4
(root-dev) GPU-ID: NVIDIA A100-SXM4-40GB
pi_die: failed cuMemGetInfo() API.
terminate called without an active exception
Aborted (core dumped)

[abagusetty]

@zjin-lcf Looks like it has no issues with the rocm-5.1.0 on Crusher. Quite strange, since both the plugins use the similar vendor APIs to get the free-memory. Which begs the question if it is related to CUDA-version supported ?

Number of Root GPUs: 2
(root-dev) GPU-ID: gfx90a:sramecc+:xnack-
(root-dev) TotalMem (bytes): 68702699520, FreeMem (bytes): 68702699520
(root-dev) GPU-ID: gfx90a:sramecc+:xnack-
(root-dev) TotalMem (bytes): 68702699520, FreeMem (bytes): 68702699520