Misc. bug: `llama_supports_gpu_offload` SIGABRT for older Vulkan versions

[DmyMi]

Name and Version

Version: b6527 (commit 7f766929)

Operating systems

Other? (Please let us know in description)

Which llama.cpp modules do you know to be affected?

libllama (core library)

Command line

Problem description & steps to reproduce

Operating System

• Android (but applicable for all, as the check is not platform specific)

Problem description

When running the application on older Android 12 (Samsung) device, I want to check if it can handle GPU, if not - fallback to CPU.

So llama_supports_gpu_offload is called, with expected result: it returns false for unsupported GPU.

Actual result: instead of getting false, i get SIGABRT and application crash.

Steps to reproduce

1. Build llama.cpp for Android with Vulkan support (cmake arguments reference). But technically you can build it for any system with older Vulkan drivers.

-DCMAKE_TOOLCHAIN_FILE=/ndkDirectory/absolutePath/build/cmake/android.toolchain.cmake \
-DGGML_LLAMAFILE=OFF \
-DGGML_OPENMP=OFF \
-DBUILD_SHARED_LIBS=ON \
-DANDROID_PLATFORM=android-28 \
-DANDROID_ABI=arm64-v8a \
-DCMAKE_SYSROOT=/ndkDirectory/absolutePath/toolchains/llvm/prebuilt/$ndkPlatform-$osArch/sysroot \
-DGGML_VULKAN=ON \
-DVulkan_INCLUDE_DIR=/absolutePath/to/Vulkan-Headers/include \
-DVulkan_LIBRARY=/ndkDirectory/absolutePath/toolchains/llvm/prebuilt/$ndkPlatform-$osArch/sysroot/usr/lib/aarch64-linux-android/28/libvulkan.so \
-DVulkan_GLSLC_EXECUTABLE=/ndkDirectory/absolutePath/shader-tools/$ndkPlatform-$osArch/glslc \
-DGGML_VULKAN_RUN_TESTS=OFF

2. Use NDK tools to put all .so into APK

3. Call llama_supports_gpu_offload on device with Vulkan version < 1.2

4. Get SIGABRT

Proposed fix

Changing how version check is handled to error instead of GGML_ABORT fixes the behaviour for me.

--- a/ggml/src/ggml-vulkan/ggml-vulkan.cpp
+++ b/ggml/src/ggml-vulkan/ggml-vulkan.cpp
@@ -4438,10 +4438,11 @@ static void ggml_vk_instance_init() {
-    if (api_version < VK_API_VERSION_1_2) {
-        std::cerr << "ggml_vulkan: Error: Vulkan 1.2 required." << std::endl;
-        GGML_ABORT("fatal error");
-    }
+    if (api_version < VK_API_VERSION_1_2) {
+        std::cerr << "ggml_vulkan: Error: Vulkan 1.2 required." << std::endl;
+        // Do not abort the whole process. Report initialization failure so the backend can be skipped.
+        throw vk::SystemError(vk::Result::eErrorFeatureNotPresent, "Vulkan 1.2 required");
+    }

and optionally (if we're not aborting entire Vulkan init process), we can catch all exceptions here and return null. Or don't and let those be handled upstream like they were before.

--- a/ggml/src/ggml-vulkan/ggml-vulkan.cpp
+++ b/ggml/src/ggml-vulkan/ggml-vulkan.cpp
@@ -12676,9 +12676,16 @@ ggml_backend_reg_t ggml_backend_vk_reg() {
     try {
         ggml_vk_instance_init();
         return ®
     } catch (const vk::SystemError& e) {
         VK_LOG_DEBUG("ggml_backend_vk_reg() -> Error: System error: " << e.what());
         return nullptr;
+    } catch (const std::exception &e) {
+        VK_LOG_DEBUG("ggml_backend_vk_reg() -> Error: " << e.what());
+        return nullptr;
+    } catch (...) {
+        VK_LOG_DEBUG("ggml_backend_vk_reg() -> Error: unknown exception during Vulkan init");
+        return nullptr;
     }
 }

First Bad Commit

No response

Relevant log output

Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 10254 (package.debug), pid 10254 (package.debug)
Cmdline: package.debug
pid: 10254, tid: 10254, name: package.debug  >>> package.debug <<<
  #01 pc 000000000004d8ac  /data/app/~~deSaqF66txhOZSY5wY_usA==/package.debug-QKqnGXuUyAClZqLnveJwJg==/base.apk!libggml-base.so (ggml_abort+228) (BuildId: b0ced8108fc489dd8d874befb5b0f42b2e44c159)
  #02 pc 0000000000091e34  /data/app/~~deSaqF66txhOZSY5wY_usA==/package.debug-QKqnGXuUyAClZqLnveJwJg==/base.apk!libggml-vulkan.so (ggml_vk_instance_init()+22600) (BuildId: 158e5e63709afcc3d3ecba0c11d2bfcd1ce761bc)
  #03 pc 0000000000098668  /data/app/~~deSaqF66txhOZSY5wY_usA==/package.debug-QKqnGXuUyAClZqLnveJwJg==/base.apk!libggml-vulkan.so (ggml_backend_vk_reg+8) (BuildId: 158e5e63709afcc3d3ecba0c11d2bfcd1ce761bc)
  #04 pc 000000000000d698  /data/app/~~deSaqF66txhOZSY5wY_usA==/package.debug-QKqnGXuUyAClZqLnveJwJg==/base.apk!libggml.so (ggml_backend_registry::ggml_backend_registry()+48) (BuildId: ede3985ef00922efd5955e60b39a4de205ed1133)
  #05 pc 000000000000b740  /data/app/~~deSaqF66txhOZSY5wY_usA==/package.debug-QKqnGXuUyAClZqLnveJwJg==/base.apk!libggml.so (ggml_backend_dev_by_type+144) (BuildId: ede3985ef00922efd5955e60b39a4de205ed1133)
  #06 pc 00000000000988dc  /data/app/~~deSaqF66txhOZSY5wY_usA==/package.debug-QKqnGXuUyAClZqLnveJwJg==/base.apk!libllama.so (llama_supports_gpu_offload+20) (BuildId: 15b1e6c0bee2b15436b4813f826f7ae162da9ce2)