Skip to content

server : handle content array in chat API #8449

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jul 12, 2024
Merged

server : handle content array in chat API #8449

merged 2 commits into from
Jul 12, 2024

Conversation

@ggerganov
Copy link
Member

@ggerganov ggerganov commented Jul 12, 2024 •
edited
Loading

fix #8367

Testing:

make -j && ./llama-server --hf-repo "Qwen/Qwen2-0.5B-Instruct-GGUF" --hf-file qwen2-0_5b-instruct-q2_k.gguf

curl \
  --request POST --url http://localhost:8080/v1/chat/completions \
  --header "Content-Type: application/json" \
  --data '{"temperature": 0.0, "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": [ { "type": "text", "text": "tell me a " }, { "type": "text", "text": "riddle" } ] } ] }'

ngxson
ngxson reviewed Jul 12, 2024
View reviewed changes
@ggerganov @ngxson
Update examples/server/utils.hpp
8428a98 
Co-authored-by: Xuan Son Nguyen <[email protected]>
@ggerganov ggerganov merged commit 4e24cff into master Jul 12, 2024
@ggerganov ggerganov deleted the gg/server-chat-content branch July 12, 2024 11:48
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Jul 13, 2024
@ggerganov @ngxson
server : handle content array in chat API (ggml-org#8449)
cd3db81 
* server : handle content array in chat API

* Update examples/server/utils.hpp

Co-authored-by: Xuan Son Nguyen <[email protected]>

---------

Co-authored-by: Xuan Son Nguyen <[email protected]>
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Jul 13, 2024
@ggerganov @ngxson
server : handle content array in chat API (ggml-org#8449)
07d457b 
* server : handle content array in chat API

* Update examples/server/utils.hpp

Co-authored-by: Xuan Son Nguyen <[email protected]>

---------

Co-authored-by: Xuan Son Nguyen <[email protected]>
Nexesenex pushed a commit to Nexesenex/croco.cpp that referenced this pull request Jul 15, 2024
@ggerganov @ngxson @Nexesenex
server : handle content array in chat API (ggml-org#8449)
3f77bcb 
* server : handle content array in chat API

* Update examples/server/utils.hpp

Co-authored-by: Xuan Son Nguyen <[email protected]>

---------

Co-authored-by: Xuan Son Nguyen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ngxson ngxson ngxson approved these changes

Assignees

No one assigned

Labels

examples server

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

server : support content array in OAI chat API

3 participants

@ggerganov @ngxson