Skip to content

Conversation

jeffbolznv
Copy link
Collaborator

The "Clamp" spec constant is already based on whether KV is a multiple of Bc, so use that to control whether bounds checking is performed. Add bounds checking to the scalar and coopmat1 paths. Coopmat2 didn't need any changes (the K/V tensors are already optionally clamped, nothing else needed to be changed).

For #16148.

@jeffbolznv jeffbolznv requested a review from 0cc4m as a code owner September 21, 2025 22:46
@github-actions github-actions bot added Vulkan Issues specific to the Vulkan backend ggml changes relating to the ggml tensor library for machine learning labels Sep 21, 2025
@jeffbolznv
Copy link
Collaborator Author

These accelerated vulkan CI jobs are a pleasant surprise. The failures are unrelated, but I'll look into them.

@0cc4m
Copy link
Collaborator

0cc4m commented Sep 27, 2025

Not the fault of this PR, but I'm getting a validation issue with Flash Attention:

VUID-VkComputePipelineCreateInfo-layout-07991(ERROR / SPEC): msgNum: 989008490 - Validation Error: [ VUID-VkComputePipelineCreateInfo-layout-07991 ] Object 0: handle = 0x2cfba2000000001c, type = VK_OBJECT_TYPE_SHADER_MODULE; Object 1: handle = 0x908683000000001d, type = VK_OBJECT_TYPE_PIPELINE_LAYOUT; | MessageID = 0x3af3126a | vkCreateComputePipe
lines(): pCreateInfos[0].stage SPIR-V (VK_SHADER_STAGE_COMPUTE_BIT) uses descriptor [Set 0, Binding 1] with a VkDescriptorSetLayoutBinding::descriptorCount of 1, but requires at least 2 in the SPIR-V.
The Vulkan spec states: If a resource variable is declared in a shader as an array, the corresponding descriptor set in layout must match the descriptor count (https://docs.vulkan.org/spec/latest/chapters/pipelines.html#VUID-VkComputePipelineCreateInfo-layout-07991)
    Objects: 2
        [0] 0x2cfba2000000001c, type: 15, name: NULL
        [1] 0x908683000000001d, type: 17, name: NULL

The "Clamp" spec constant is already based on whether KV is a multiple of Bc,
so use that to control whether bounds checking is performed. Add bounds checking
to the scalar and coopmat1 paths. Coopmat2 didn't need any changes (the K/V
tensors are already optionally clamped, nothing else needed to be changed).
@0cc4m 0cc4m merged commit e6d65fb into ggml-org:master Sep 27, 2025
54 of 56 checks passed
netrunnereve pushed a commit to netrunnereve/llama.cpp that referenced this pull request Oct 2, 2025
…6160)

The "Clamp" spec constant is already based on whether KV is a multiple of Bc,
so use that to control whether bounds checking is performed. Add bounds checking
to the scalar and coopmat1 paths. Coopmat2 didn't need any changes (the K/V
tensors are already optionally clamped, nothing else needed to be changed).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ggml changes relating to the ggml tensor library for machine learning Vulkan Issues specific to the Vulkan backend
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants