[ML][Inference] Adjust inference configuration option API #47812
Conversation
|
Pinging @elastic/ml-core (:ml) |
benwtrent
force-pushed
the
feature/ml-inference-make-params-model-type-specific
branch
from
3286dfc to
77f11ca
Compare
droberts195
left a comment
droberts195
left a comment
There was a problem hiding this comment.
This looks like a good strategy for providing extensibility in the future.
Since these objects could eventually be stored in cluster state via the ingest pipeline definitions I am just wondering if this is opening up the possibility of unparseable cluster state if new types of inference config are added via newer nodes in mixed version clusters.
One possible strategy for defending against this future problem could be to change isTargetTypeSupported(TargetType targetType) to isTargetTypeSupportedAtVersion(TargetType targetType, Version version). When called we'd pass the minimum node version in the cluster to that method rather than the current node version. Then we could defend against people configuring new types of inference model until every node in the cluster was on the required version for that type of model.
Another option might be to simply have a minimumSupportedVersion() method in the InferenceConfig interface and check that at the points where we might get such a config from a user.
Or maybe you already have another plan for defending against mixed version cluster problems.
But I think we need something for this.
|
Thanks for the feedback!
When the pipeline & processor config is stored in cluster state, it is a simple So, there are no worries about breaking the cluster. The worse thing is that an unsupported node version reads in the config, but fails to create the processor. |
benwtrent
deleted the
feature/ml-inference-make-params-model-type-specific
branch
* [ML][Inference] adds lazy model loader and inference (#47410) This adds a couple of things: - A model loader service that is accessible via transport calls. This service will load in models and cache them. They will stay loaded until a processor no longer references them - A Model class and its first sub-class LocalModel. Used to cache model information and run inference. - Transport action and handler for requests to infer against a local model Related Feature PRs: * [ML][Inference] Adjust inference configuration option API (#47812) * [ML][Inference] adds logistic_regression output aggregator (#48075) * [ML][Inference] Adding read/del trained models (#47882) * [ML][Inference] Adding inference ingest processor (#47859) * [ML][Inference] fixing classification inference for ensemble (#48463) * [ML][Inference] Adding model memory estimations (#48323) * [ML][Inference] adding more options to inference processor (#48545) * [ML][Inference] handle string values better in feature extraction (#48584) * [ML][Inference] Adding _stats endpoint for inference (#48492) * [ML][Inference] add inference processors and trained models to usage (#47869) * [ML][Inference] add new flag for optionally including model definition (#48718) * [ML][Inference] adding license checks (#49056) * [ML][Inference] Adding memory and compute estimates to inference (#48955)
* [ML][Inference] adds lazy model loader and inference (elastic#47410) This adds a couple of things: - A model loader service that is accessible via transport calls. This service will load in models and cache them. They will stay loaded until a processor no longer references them - A Model class and its first sub-class LocalModel. Used to cache model information and run inference. - Transport action and handler for requests to infer against a local model Related Feature PRs: * [ML][Inference] Adjust inference configuration option API (elastic#47812) * [ML][Inference] adds logistic_regression output aggregator (elastic#48075) * [ML][Inference] Adding read/del trained models (elastic#47882) * [ML][Inference] Adding inference ingest processor (elastic#47859) * [ML][Inference] fixing classification inference for ensemble (elastic#48463) * [ML][Inference] Adding model memory estimations (elastic#48323) * [ML][Inference] adding more options to inference processor (elastic#48545) * [ML][Inference] handle string values better in feature extraction (elastic#48584) * [ML][Inference] Adding _stats endpoint for inference (elastic#48492) * [ML][Inference] add inference processors and trained models to usage (elastic#47869) * [ML][Inference] add new flag for optionally including model definition (elastic#48718) * [ML][Inference] adding license checks (elastic#49056) * [ML][Inference] Adding memory and compute estimates to inference (elastic#48955)
* [ML] ML Model Inference Ingest Processor (#49052) * [ML][Inference] adds lazy model loader and inference (#47410) This adds a couple of things: - A model loader service that is accessible via transport calls. This service will load in models and cache them. They will stay loaded until a processor no longer references them - A Model class and its first sub-class LocalModel. Used to cache model information and run inference. - Transport action and handler for requests to infer against a local model Related Feature PRs: * [ML][Inference] Adjust inference configuration option API (#47812) * [ML][Inference] adds logistic_regression output aggregator (#48075) * [ML][Inference] Adding read/del trained models (#47882) * [ML][Inference] Adding inference ingest processor (#47859) * [ML][Inference] fixing classification inference for ensemble (#48463) * [ML][Inference] Adding model memory estimations (#48323) * [ML][Inference] adding more options to inference processor (#48545) * [ML][Inference] handle string values better in feature extraction (#48584) * [ML][Inference] Adding _stats endpoint for inference (#48492) * [ML][Inference] add inference processors and trained models to usage (#47869) * [ML][Inference] add new flag for optionally including model definition (#48718) * [ML][Inference] adding license checks (#49056) * [ML][Inference] Adding memory and compute estimates to inference (#48955) * fixing version of indexed docs for model inference
4 participants
When requesting a model inference, it is helpful to know the following information early:
This change creates a new named writable that users will provide when creating an inference processor/making an API request against an existing model.
The design is meant to be extensible, and possibly have new "types" of models in the future (outside of Regression/Classification, looking at you Clustering, NLP weirdness).
Additionally, this cleans up the dependencies between the output aggregation + target type values in the Ensemble model.