Bug: Incorrect nb

[wangshuai09]

What happened?

Hi, i'm working on Support for Ascend NPU backend and found a curious nb.
Way to reproduce:

1. Model: NousResearch/Hermes-2-Pro-Llama-3-8B-GGUF
2. Build: 1cmake .. -DCMAKE_BUILD_TYPE=debug && make -j16
3. Run: ./bin/main -m /home/models/hermes_gguf/Hermes-2-Pro-Llama-3-8B-F16.gguf -p "Building a website can be done in 10 simple steps:" -ngl 0 -sm none -mg 0 -t 0
4. Result: In MUL_MATop which node id is 1110, the src0->nb is [2,2048,256,2048]. I think this is wrong because overlaped, but the model can still generatate words.
   

Name and Version

version: 2775 (ef6e54c)
built with cc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 for aarch64-linux-gnu

What operating system are you seeing the problem on?

No response

Relevant log output

No response

[ggerganov]

This is expected. This is likely the k tensor:

https://github.com/ggerganov/llama.cpp/blob/a9cae48003dfc4fe95b8f5c81682fc6e63425235/llama.cpp#L6978-L6983

It is permuted: dims (0,1,2,3) -> (0,2,1,3)

The stride for dim 3 (i.e. nb[3]) is invalid but it is never used because ne[3] is 1 (see #7856 for more info) so it's OK

[wangshuai09]

Thanks for your replay, This solved my confusion. Continue working!