Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Conversation

@nguyenhoangthuan99
Copy link
Contributor

@nguyenhoangthuan99 nguyenhoangthuan99 commented Sep 22, 2024

Replace jinja2cpp with llama.cpp for chat template parsing

Summary

This PR replaces the jinja2cpp library with code derived from the llama.cpp project for parsing chat templates. This change significantly reduces our binary size and improves the reliability of template parsing.

Changes

  • Removed jinja2cpp library and related dependencies
  • Integrated chat template parsing functionality from llama.cpp
  • Updated relevant code to use the new parsing method
  • Added appropriate license and attribution for llama.cpp code

Benefits

  1. Reduced Binary Size: Eliminates more space from our binary file.
  2. Improved Reliability: The llama.cpp parsing method has shown better consistency in handling our chat templates compared to jinja2cpp, which occasionally failed to parse correctly.
  3. Simplified Dependencies: Removes a complex external library, potentially reducing build times and complexity.

Related Issues

Fix #1063

Before add custom chat template renderer

image

After

image

@nguyenhoangthuan99 nguyenhoangthuan99 marked this pull request as ready for review September 23, 2024 01:19
Copy link
Contributor

@namchuai namchuai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ lgtm

Copy link
Contributor

@dan-menlo dan-menlo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chore: Improve package/binary size by remove jinja2

5 participants