webgpu : fix build on emscripten

[ngxson]

Ref original webgpu PR: #14978

Example command:

# install emscripten: brew install emscripten

emcmake cmake -B build-wasm -DGGML_WEBGPU=ON -DLLAMA_CURL=OFF -DGGML_WEBGPU_DEBUG=ON
cmake --build build-wasm --target test-backend-ops

[Green-Sky]

Why not just a.out* ?

[ngxson]

@ggerganov @slaren Quick question, I'm building test-backend-ops on wasm without multi-thread support for simplification. However, seems like ggml-cpu now always requires a threadpool, which makes multi-thread a requirement.

So I'm wondering, is there any ways to completely disable threadpool?

Edit: I'm referring to this code:

llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c

Lines 3124 to 3130 in c4df49a

	if (threadpool == NULL) {
	//GGML_PRINT_DEBUG("Threadpool is not specified. Will create a disposable threadpool : n_threads %d\n", n_threads);
	disposable_threadpool = true;
	struct ggml_threadpool_params ttp = ggml_threadpool_params_default(n_threads);
	threadpool = ggml_threadpool_new_impl(&ttp, cgraph, cplan);
	} else {

[ggerganov]

I think a threadpool is currently required - don't think there is an easy workaround. @max-krasnyansky any thoughts?

[ngxson]

I think a threadpool is currently required - don't think there is an easy workaround. @max-krasnyansky any thoughts?

Hmm ok that means both wllama and whisper.cpp single-thread wasm builds are currently broken. Having single-thread support would be nice, but it's not urgent though.

[ggerganov]

Yes, we should support to launch a single-thread compute without invoking synchronization primitives and spawning threads so that thread-less WASM works. Shouldn't be hard to implement.

Looking at the implementation, I think almost everything is inplace for that. Where does the single-thread WASM fail when you call ggml compute with n_threads == 1?

[ngxson]

It currently fails at this line:

llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c

Line 3091 in c4df49a

GGML_ASSERT(rc == 0);

I don't have the stack trace due to some difficulty debugging in-browser, but it's very likely invoked by this line:

llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c

Line 3129 in c4df49a

threadpool = ggml_threadpool_new_impl(&ttp, cgraph, cplan);

Where we try to create a threadpool of one single thread

[ngxson]

Also just want to note that atomic ops like atomic_store_explicit are not available in single-thread build, so this should requires some extra adaptations

[ggerganov]

It currently fails at this line:

llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c

Line 3091 in c4df49a

GGML_ASSERT(rc == 0);

I don't have the stack trace due to some difficulty debugging in-browser, but it's very likely invoked by this line:

llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c

Line 3129 in c4df49a

threadpool = ggml_threadpool_new_impl(&ttp, cgraph, cplan);

Where we try to create a threadpool of one single thread

To enter that loop, it would mean that tpp->nthreads is larger than 1, because the loop stars from 1:

    for (int j = 1; j < tpp->n_threads; j++) {
        ggml_thread_cpumask_next(tpp->cpumask, workers[j].cpumask, tpp->strict_cpu, &cpumask_iter);

        int32_t rc = ggml_thread_create(&workers[j].thrd, NULL, ggml_graph_compute_secondary_thread, &workers[j]);
        GGML_ASSERT(rc == 0);
    }

Could the calling program be using n_threads > 1 instead of n_threads == 1?

[ngxson]

Yeah you're right, the n_thread for CPU is not correctly set for test-backend-ops. I added it in bf9d14c and it's now working as expected

[reeselevine]

What's the status on this PR, and more generally, potential integration of WebGPU with wllama @ngxson?

When #16400 is merged the WebGPU backend should be able to run all the operations for a decent number of text-generation models (excepting flash attention, but my understanding is it falls back to standard multi-kernel attention for now).

There's still some work to do on optimizing the existing shaders for it to really work well, but it would be great to also start getting it ready in a demo form on the browser. Happy to help work on integration if you'd like too.

[ngxson]

hey @reeselevine I still have some weird issues where running this can cause the browser to hang. need to investigate this a bit more.