[SYCL] Persistent program cache (#3391)

First part of implementation for persistent cache of device programs following sycl/doc/KernelProgramCache.md:

Programs built for target device are stored in on-disk cache and used next time when the same application is executed.  
What is done:
 - storing and loading files when built program is not found in in-memory cache;
 - file system concurrent access management (lock file is created per cache item while data is being written, read operation does not lock the cache item);
 - thread safety for persistent device code cache is not needed as soon as build/in-memory cache part is managed by program_manager and persisten_device_code_cache does not store data shared between threads.
 - Implement support for environment variables affecting cache:
   SYCL_CACHE_DIR
   SYCL_CACHE_TRACE
   SYCL_CACHE_DISABLE_PERSISTENT
   SYCL_CACHE_MIN_DEVICE_IMAGE_SIZE
   SYCL_CACHE_MAX_DEVICE_IMAGE_SIZE
   SYCL_CACHE_MIN_DEVICE_IMAGE_SIZE
 - Errors happening during filesystem operation cause corresponding cache operation silently ends:
   - for read operation, cache miss happens (without SYCL error), message is sent to std::cerr if SYCL_CACHE_TRACE is set;
   - for write operation, no cache item stored (without SYCL error), message is sent to std::cerr if SYCL_CACHE_TRACE is set;
 - Add tests:
   - on-device tests for basic cache functionality and environment variables processing;
   - unit tests for concurrent access to filesystem and error management.
 
To be done in next patches:
 - implement rest of environment variables and extend them to both in-memory and persistent cache;
 - implement kernel eviction mechanism for on-device and in-memory caches.

Author: vladimirlaz

sycl/doc/EnvironmentVariables.md

@@ -35,6 +35,14 @@ subject to change. Do not rely on these variables in production code.
 | SYCL_DISABLE_PARALLEL_FOR_RANGE_ROUNDING | Any(\*) | Disables automatic rounding-up of parallel_for invocation ranges. |
 | SYCL_ENABLE_PCI | Integer | When set to 1, enables obtaining the GPU PCI address when using the Level Zero backend. The default is 0. |
 | SYCL_HOST_UNIFIED_MEMORY | Integer | Enforce host unified memory support or lack of it for the execution graph builder. If set to 0, it is enforced as not supported by all devices. If set to 1, it is enforced as supported by all devices. |
+| SYCL_CACHE_DIR | Path | Path to persistent cache root directory. Default values are `%AppData%\libsycl_cache` for Windows and `$XDG_CACHE_HOME/libsycl_cache` on Linux, if XDG_CACHE_HOME is not set then `$HOME/.cache/libsycl_cache`. |
+| SYCL_CACHE_TRACE | Any(\*) | Enables printing messages to std::cerr when non-blocking failures happen (e.g. unable to access cache item file). Default values if off. |
+| SYCL_CACHE_DISABLE_PERSISTENT | Any(\*) | Switches persistent cache switch off. Default value is ON. |
+| SYCL_CACHE_EVICTION_DISABLE | Any(\*) | Switches cache eviction off. Default value is ON. |
+| SYCL_CACHE_MAX_SIZE | Positive integer | Cache eviction is triggered once total size of cached images exceeds the value in megabytes (default - 8 192 for 8 GB). Set to 0 to disable size-based cache eviction. |
+| SYCL_CACHE_THRESHOLD | Positive integer | Cache eviction threshold in days (default value is 7 for 1 week). Set to 0 for disabling time-based cache eviction. |
+| SYCL_CACHE_MIN_DEVICE_IMAGE_SIZE | Positive integer | Minimum size of device code image in bytes which is reasonable to cache on disk because disk access operation may take more time than do JIT compilation for it. Default value is 0 to cache all images. |
+| SYCL_CACHE_MAX_DEVICE_IMAGE_SIZE | Positive integer | Maximum size of device image in bytes which is cached. Too big kernels may overload disk too fast. Default value is 1 GB. |
 
 `(*) Note: Any means this environment variable is effective when set to any non-null value.`
 

sycl/doc/KernelProgramCache.md

@@ -172,15 +172,8 @@ three sources of build options:
 
 ## Cache configuration
 
-There is set of configuration parameters which can be set as environment variables or parameters in `sycl.conf`  and affect cache behavior:
-| Environment variable | Values | Description |
-| -------------------- | ------ | ----------- |
-| `SYCL_CACHE_DIR`| Path | Path to persistent cache root directory. Default values are `%AppData%\Intel\sycl_program_cache` for Windows and `$HOME/intel/sycl_program_cache` on Linux. |
-| `SYCL_CACHE_ENABLED` | ON, OFF | Switches persistent cache switch  on or off. Default value is ON. |
-| `SYCL_CACHE_MAX_SIZE` | Positive integer | Cache eviction is triggered once total size of cached images exceeds the value in megabytes (default - 8 192 for 8 GB). Set to 0 to disable size-based cache eviction. |
-| `SYCL_CACHE_THRESHOLD` | Positive integer | Cache eviction threshold in days (default value is 7 for 1 week). Set to 0 for disabling time-based cache eviction. |
-| `SYCL_CACHE_MIN_DEVICE_IMAGE_SIZE` | Positive integer | Minimum size of device code image in kilobytes which is reasonable to cache on disk because disk access operation may take more time than do JIT compilation for it. Default value is 0 to cache all images. |
-| `SYCL_CACHE_MAX_DEVICE_IMAGE_SIZE` | Positive integer | Maximum size of device image in megabytes which is cached. Too big kernels may overload disk too fast. Default value is 0 to cache all images. |
+The environment variables which affect cache behavior are described in
+[EnvironmentVariables.md](https://github.com/intel/llvm/blob/sycl/sycl/doc/EnvironmentVariables.md).
 
 ## Implementation details
 

sycl/include/CL/sycl/detail/os_util.hpp

@@ -16,6 +16,8 @@
 #include <cstdint>
 #include <cstdlib>
 #include <string>
+#include <sys/stat.h>
+#include <sys/types.h>
 
 #ifdef _WIN32
 #define __SYCL_RT_OS_WINDOWS
@@ -80,6 +82,20 @@ class __SYCL_EXPORT OSUtil {
 
   /// Deallocates the memory referenced by \p Ptr.
   static void alignedFree(void *Ptr);
+
+  /// Make directory recursively and returns zero code on success
+  static int makeDir(const char *Dir);
+
+  /// Checks if specified path is present
+  static inline bool isPathPresent(const std::string &Path) {
+#ifdef __SYCL_RT_OS_WINDOWS
+    struct _stat Stat;
+    return !_stat(Path.c_str(), &Stat);
+#else
+    struct stat Stat;
+    return !stat(Path.c_str(), &Stat);
+#endif
+  }
 };
 
 } // namespace detail

sycl/source/CMakeLists.txt

@@ -137,6 +137,7 @@ set(SYCL_SOURCES
     "detail/queue_impl.cpp"
     "detail/online_compiler/online_compiler.cpp"
     "detail/os_util.cpp"
+    "detail/persistent_device_code_cache.cpp"
     "detail/platform_util.cpp"
     "detail/reduction.cpp"
     "detail/sampler_impl.cpp"

sycl/source/detail/config.def

@@ -20,3 +20,12 @@ CONFIG(SYCL_DEVICE_FILTER, 1024, __SYCL_DEVICE_FILTER)
 CONFIG(SYCL_PROGRAM_LINK_OPTIONS, 64, __SYCL_PROGRAM_LINK_OPTIONS)
 CONFIG(SYCL_PROGRAM_COMPILE_OPTIONS, 64, __SYCL_PROGRAM_COMPILE_OPTIONS)
 CONFIG(SYCL_HOST_UNIFIED_MEMORY, 1, __SYCL_HOST_UNIFIED_MEMORY)
+// 260 (Windows limit) - 12 (filename) - 84 (cache directory structure)
+CONFIG(SYCL_CACHE_DIR, 164, __SYCL_CACHE_DIR)
+CONFIG(SYCL_CACHE_TRACE, 1, __SYCL_CACHE_TRACE)
+CONFIG(SYCL_CACHE_DISABLE_PERSISTENT, 1, __SYCL_CACHE_DISABLE_PERSISTENT)
+CONFIG(SYCL_CACHE_EVICTION_DISABLE, 1, __SYCL_CACHE_EVICTION_DISABLE)
+CONFIG(SYCL_CACHE_MAX_SIZE, 16, __SYCL_CACHE_MAX_SIZE)
+CONFIG(SYCL_CACHE_THRESHOLD, 16, __SYCL_CACHE_THRESHOLD)
+CONFIG(SYCL_CACHE_MIN_DEVICE_IMAGE_SIZE, 16, __SYCL_CACHE_MIN_DEVICE_IMAGE_SIZE)
+CONFIG(SYCL_CACHE_MAX_DEVICE_IMAGE_SIZE, 16, __SYCL_CACHE_MAX_DEVICE_IMAGE_SIZE)

sycl/source/detail/os_util.cpp

@@ -24,11 +24,13 @@
 #include <libgen.h> // for dirname
 #include <link.h>
 #include <linux/limits.h> // for PATH_MAX
+#include <sys/stat.h>
 #include <sys/sysinfo.h>
 
 #elif defined(__SYCL_RT_OS_WINDOWS)
 
 #include <Windows.h>
+#include <direct.h>
 #include <malloc.h>
 #include <shlwapi.h>
 
@@ -211,6 +213,19 @@ std::string OSUtil::getCurrentDSODir() {
   return Path;
 }
 
+std::string OSUtil::getDirName(const char *Path) {
+  std::string Tmp(Path);
+  // Remove trailing directory separators
+  Tmp.erase(Tmp.find_last_not_of("/\\") + 1, std::string::npos);
+
+  int pos = Tmp.find_last_of("/\\");
+  if (pos != std::string::npos)
+    return Tmp.substr(0, pos);
+
+  // If no directory separator is present return initial path like dirname does
+  return Tmp;
+}
+
 #elif defined(__SYCL_RT_OS_DARWIN)
 OSModuleHandle OSUtil::getOSModuleHandle(const void *VirtAddr) {
   Dl_info Res;
@@ -258,6 +273,32 @@ void OSUtil::alignedFree(void *Ptr) {
 #endif
 }
 
+/* This is temporary solution until std::filesystem is available when SYCL RT
+ * is moved to c++17 standard*/
+
+/* Create directory recursively and return non zero code on success*/
+int OSUtil::makeDir(const char *Dir) {
+  assert((Dir != nullptr) && "Passed null-pointer as directory name.");
+  if (isPathPresent(Dir))
+    return 0;
+
+  std::string Path{Dir}, CurPath;
+  size_t pos = 0;
+
+  do {
+    pos = Path.find_first_of("/\\", ++pos);
+    CurPath = Path.substr(0, pos);
+#if defined(__SYCL_RT_OS_LINUX)
+    auto Res = mkdir(CurPath.c_str(), 0777);
+#else
+    auto Res = _mkdir(CurPath.c_str());
+#endif
+    if (Res && errno != EEXIST)
+      return Res;
+  } while (pos != std::string::npos);
+  return 0;
+}
+
 } // namespace detail
 } // namespace sycl
 } // __SYCL_INLINE_NAMESPACE(cl)