Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Commit 87ad9bc

Browse files
committed
feat: add new parameters and flags to local engine configuration
This commit introduces new configuration parameters and their corresponding command-line flags for the local engine. The changes include: - Adding "flash_attn" to ignored parameters - Mapping UI parameters to CLI flags (e.g., cpu_threads → --threads) - Expanding support for various model configuration options These additions enhance the flexibility of the local engine by enabling fine-grained control over performance and behavior through both UI and CLI interfaces.
1 parent 41023d3 commit 87ad9bc

File tree

1 file changed

+11
-0
lines changed

1 file changed

+11
-0
lines changed

engine/extensions/local-engine/local_engine.cc

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,20 @@ const std::unordered_set<std::string> kIgnoredParams = {
2222
"user_prompt", "min_keep", "mirostat", "mirostat_eta",
2323
"mirostat_tau", "text_model", "version", "n_probs",
2424
"object", "penalize_nl", "precision", "size",
25+
"flash_attn",
2526
"stop", "tfs_z", "typ_p", "caching_enabled"};
2627

2728
const std::unordered_map<std::string, std::string> kParamsMap = {
29+
{"cpu_threads", "--threads"},
30+
{"n_ubatch", "--ubatch-size"},
31+
{"n_batch", "--batch-size"},
32+
{"n_parallel", "--parallel"},
33+
{"temperature", "--temp"},
34+
{"top_k", "--top-k"},
35+
{"top_p", "--top-p"},
36+
{"min_p", "--min-p"},
37+
{"dynatemp_exponent", "--dynatemp-exp"},
38+
{"ctx_len", "--ctx-size"},
2839
{"ngl", "-ngl"},
2940
};
3041

0 commit comments

Comments
 (0)