webui : display prompt timing stats

[ggerganov]

Great. Btw, we could also show the stats for the prompt, not just for the generated tokens. This ok for now - we can expand this information in the future.

Originally posted by @ggerganov in #16901 (review)

Display the prompt_ stats in a similar way as the predicted_ stats:

llama.cpp/tools/server/server.cpp

Lines 639 to 651 in 4621d99

	json base = {
	{"cache_n", cache_n},
	{"prompt_n", prompt_n},
	{"prompt_ms", prompt_ms},
	{"prompt_per_token_ms", prompt_per_token_ms},
	{"prompt_per_second", prompt_per_second},
	{"predicted_n", predicted_n},
	{"predicted_ms", predicted_ms},
	{"predicted_per_token_ms", predicted_per_token_ms},
	{"predicted_per_second", predicted_per_second},
	};

[ServeurpersoCom]

You’re suggesting enriching the OpenAI-Compat chunks with backend metrics, essentially injecting the prompt-phase stats into the streamed data? We need to find the right balance here: either repeat the data in each chunk or expose it through a custom endpoint, without breaking OAI compatibility.

[ggerganov]

Haven't thought this through in details - I actually thought that the prompt processing stats were already included in the responses and it's just a matter of the WebUI visualizing them. If it's not the case, then ignore my proposal.

expose it through a custom endpoint, without breaking OAI compatibility.

Sounds like a good idea.