epic: SQLite database implementation

[nguyenhoangthuan99]

Implement a ModelListUtils class base on this discussion #1154

Overview

The ModelListUtils class provides a robust and thread-safe interface for managing a list of machine learning models. It handles operations such as adding, updating, deleting, and retrieving model information, with a focus on maintaining unique identifiers for each model.

Key Features

1. Thread-safe Operations: Uses mutex locking to ensure thread safety for all operations.
2. File-based Storage: Manages model information in a text file, allowing for persistence between program runs.
3. Unique Identifier Generation: Automatically generates unique, shortened aliases for models.
4. CRUD Operations: Supports Creating, Reading, Updating, and Deleting model entries.
5. Error Handling: Robust error checking and exception throwing for invalid operations.

Main Components

ModelEntry Struct

Represents a single model entry with fields:

• model_id: Unique identifier for the model
• author_repo_id: Author or repository identifier
• branch_name: Branch name in the repository
• path_to_model_yaml: Path to the model's YAML file
• model_alias: Shortened alias for the model
• status: Current status of the model (READY or RUNNING)

ModelListUtils Class Methods

1. LoadModelList()

   • Loads model entries from the file, creating the file if it doesn't exist.
   • Returns a vector of ModelEntry objects.

2. SaveModelList()

   • Saves the current list of model entries to the file.

3. GetModelInfo(identifier)

   • Retrieves model information based on model_id or model_alias.
   • Throws an exception if the model is not found.

4. AddModelEntry(new_entry, use_short_alias)

   • Adds a new model entry to the list.
   • Optionally generates a shortened alias for the model.
   • Ensures uniqueness of model_id and model_alias.

5. UpdateModelEntry(identifier, updated_entry)

   • Updates an existing model entry.
   • Identified by either model_id or model_alias.

6. DeleteModelEntry(identifier)

   • Deletes a model entry if it exists and is in READY state.

7. PrintModelInfo(entry)

   • Prints detailed information about a model entry.

8. GenerateShortenedAlias(model_id, entries)

   • Generates a unique, shortened alias for a model.
   • Uses a hierarchical approach to create aliases, prioritizing brevity while ensuring uniqueness.

Alias Generation Logic

The GenerateShortenedAlias function creates aliases in the following order:

1. Filename only (e.g., model_id_xxx)
2. Parent directory + filename (e.g., llama3.1-7b-gguf:model_id_xxx)
3. Grandparent directory + parent directory + filename (e.g., bartowski:llama3.1-7b-gguf/model_id_xxx)
4. Full path (e.g., huggingface.co:bartowski/llama3.1-7b-gguf/model_id_xxx)

It returns the shortest unique alias, appending a numeric suffix if necessary to ensure uniqueness.

[freelerobot]

Problem

• Cortex needs a model index to know which models are available (and if the model has been correctly configured)

Success Criteria

• models.list does not duplicate information in model.yaml
  • models.list has a clear separate purpose than the yaml.
  • duplicating fields across list and yaml introduces edge cases & unmaintainability
• cortex pull/run correctly updates models.list
• cortex import correctly updates models.list

Related

• feat: Cortex model importing #1058
• epic: Implement new Model Folder and model.yaml #1154

Questions

1. Naming: models.list or model.list? The former is more grammatically correct.
2. Why duplicate fields like author_repo_id, branch_name across both model.yaml and models.list? Should we move fields from model.yaml into list instead? OR vice versa?
3. Is there a better way to track model status than to do file IO during runtime?
4. Should we version the models.list? i.e. if we change the schema in the future, it is trackable.
5. Example: user symlinks a local model: llama3 local imported seems redundant

llama3 local imported /Users/nicolezhu/cortexcpp-nightly/models/imported/llama3.yml llama3 READY

6. @louis-jan mentioned using a db at one point. Does this beg that question?
7. When is model.list first created? Doesn't seem to be upon installation. Related: bug: Fail to get list model information: Unable to create model.list file #1285

Tldr

At the moment, the current models.list design is complicated. I am worried ab out the long term maintainability and edge cases.

1. Should model.list just be a bloody database?
2. What is the separation of concerns btw this and model.yaml (trying our best to not duplicate fields)

My 2cents:

• models.list: tracks model metadata needed for locally managing models, specific to cortex. e.g. id, alias, yaml path, maybe binary path
• model.yaml: tracks runtime parameters needed for running models, NOT specific to cortex runtime, e.g. source url, model name, all hyperparams...

cc @dan-homebrew

[freelerobot]

cc @louis-jan to take a look and lets discuss here 🙏