Is possible to hide start / stop tokens?

[mirek190]

Example llamacpp output in the terminal with llama3

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a helpful, smart, kind, and efficient AI assistant. You always fulfill the user's requests to the best of your ability.
> <|start_header_id|>user<|end_header_id|>

name countries starts with letter Z. \
<|start_header_id|>assistant<|end_header_id|>

Here are some countries that start with the letter Z:

1. Zambia
2. Zimbabwe

Note: These are the only two countries whose names begin with the letter Z.

Is there anything else I can help you with?<|eot_id|>

> <|start_header_id|>user<|end_header_id|>

Is possible to hide system, start, stop, in-prefix and in-suffif tokens in the terminal ?

[dawidpotocki]

For the system message you can do --no-display-prompt.

As far as I know, for the rest you need to patch llama.cpp.

Here is a simple patch I wrote for my own use:
dawidpotocki@8c2db37

Example output:

> Hello!
Hello! It's nice to meet you. Is there something I can help you with, or would you like to chat?

> Nah, I'm good.
No worries then! It's always okay to just chill and say hi.

>

[mirek190]

wow nice!

That should be added to the main.

[arch-btw]

@dawidpotocki that's perfect! I agree, would you please consider adding it to main? Thank you!

[MB7979]

Yes, the printing of the EOS token bugs me too. I’m only used to that when tokenisation is awry, so it feels strange to have it be a default in main.