Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Conversation

@nguyenhoangthuan99
Copy link
Contributor

@nguyenhoangthuan99 nguyenhoangthuan99 commented Sep 23, 2024

Closes #1154

command option:
image
Now cortex.cpp can update model parameters by passing it to this command, only passed argument will be updated

If successfully Update, it will output which parameters of model ID is updated

image

@nguyenhoangthuan99 nguyenhoangthuan99 requested review from dan-menlo, namchuai and vansangpfiev and removed request for namchuai and vansangpfiev September 23, 2024 10:37
@nguyenhoangthuan99 nguyenhoangthuan99 marked this pull request as ready for review September 23, 2024 14:28
@dan-menlo
Copy link
Contributor

@nguyenhoangthuan99 Can I check the main purpose of the cortex models update is for Jan to be able update model parameters in model.yaml?

  • In the future, we will need clear differentiation by model engine, as the parameters differ by each engine


void ModelUpdCmd::UpdateConfig(const std::string& key,
const std::string& value) {
static const std::unordered_map<
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I think there's a better way. We can define a function: UpdateModelConfig(const std::string& attr, std::variant<std::string, explicit_int, explicit_bool>) and handle the update logic there.

However if this works for you then I'm fine with it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think should keep it this way. We don't know which format user will input, so need to handle it for specific cases

@nguyenhoangthuan99 nguyenhoangthuan99 merged commit fb55754 into feat/new-model-folder Sep 24, 2024
@nguyenhoangthuan99 nguyenhoangthuan99 deleted the feat/model-update-cmd branch September 24, 2024 05:28
nguyenhoangthuan99 added a commit that referenced this pull request Sep 24, 2024
* feat: pulling interact with new model.list

* feat: return model id when download model success

* feat: model delete for new model.list (#1317)

* feat: models delete for new models data folder structure

* feat: delete model

* feat: cortex chat/run/models start with new model data structure (#1301)

* feat: cortex chat/run/models start with new model data structure

* temp

* fix: use model_id from model service and std::get_line

* f:m

* Model update command/api (#1309)

* Model update command/api

* fix: resume download failed

Signed-off-by: James <[email protected]>

* fix: align github syntax for cuda (#1316)

* fix: require sudo for cortex update (#1318)

* fix: require sudo for cortex update

* fix: comment

* refactor code

* Format code

* Add clean up when finish test

* remove model.list after finish test

* Fix windows CI build

---------

Signed-off-by: James <[email protected]>
Co-authored-by: James <[email protected]>
Co-authored-by: vansangpfiev <[email protected]>

* Add more fields to handle when update

---------

Signed-off-by: James <[email protected]>
Co-authored-by: James <[email protected]>
Co-authored-by: vansangpfiev <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants