Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,12 @@ the `model_snapshot_id` in the results from the get jobs API.
==== {api-path-parms-title}

`<job_id>`::
(Required, string) Identifier for the job.
(Required, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]

`<snapshot_id>`::
(Required, string) Identifier for the model snapshot.
(Required, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=snapshot-id]

[[ml-delete-snapshot-example]]
==== {api-examples-title}
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/ml/anomaly-detection/apis/get-job.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ value.

`model_snapshot_id`::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=model-snapshot-id]
include::{docdir}/ml/ml-shared.asciidoc[tag=snapshot-id]

[[ml-get-job-response-codes]]
==== {api-response-codes-title}
Expand Down
152 changes: 121 additions & 31 deletions docs/reference/ml/anomaly-detection/apis/get-snapshot.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,13 @@ Retrieves information about model snapshots.
include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]

`<snapshot_id>`::
(Optional, string) Identifier for the model snapshot. If you do not specify
this optional parameter, the API returns information about all model snapshots.
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=snapshot-id]
+
--
If you do not specify this optional parameter, the API returns information about
all model snapshots.
--

[[ml-get-snapshot-request-body]]
==== {api-request-body-title}
Expand All @@ -58,52 +63,137 @@ include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]
[[ml-get-snapshot-results]]
==== {api-response-body-title}

The API returns the following information:
The API returns an array of model snapshot objects, which have the following
properties:

`model_snapshots`::
(array) An array of model snapshot objects. For more information, see
<<ml-snapshot-resource>>.
`description`::
(string) An optional description of the job.

`job_id`::
(string) A numerical character string that uniquely identifies the job that the
snapshot was created for.

`latest_record_time_stamp`::
(date) The timestamp of the latest processed record.

`latest_result_time_stamp`::
(date) The timestamp of the latest bucket result.

`min_version`::
(string) The minimum version required to be able to restore the model snapshot.

`model_size_stats`::
(object) Summary information describing the model.

`model_size_stats`.`bucket_allocation_failures_count`:::
(long) The number of buckets for which entities were not processed due to memory
limit constraints.

`model_size_stats`.`job_id`:::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]

`model_size_stats`.`log_time`:::
(date) The timestamp that the `model_size_stats` were recorded, according to
server-time.

`model_size_stats`.`memory_status`:::
(string) The status of the memory in relation to its `model_memory_limit`.
Contains one of the following values.
+
--
* `hard_limit`: The internal models require more space that the configured
memory limit. Some incoming data could not be processed.
* `ok`: The internal models stayed below the configured value.
* `soft_limit`: The internal models require more than 60% of the configured
memory limit and more aggressive pruning will be performed in order to try to
reclaim space.

--

`model_size_stats`.`model_bytes`:::
(long) An approximation of the memory resources required for this analysis.

`model_size_stats`.`model_bytes_exceeded`:::
(long) The number of bytes over the high limit for memory usage at the last allocation failure.

`model_size_stats`.`model_bytes_memory_limit`:::
(long) The upper limit for memory usage, checked on increasing values.

`model_size_stats`.`result_type`:::
(string) Internal. This value is always set to "model_size_stats".

`model_size_stats`.`timestamp`:::
(date) The timestamp that the `model_size_stats` were recorded, according to the
bucket timestamp of the data.

`model_size_stats`.`total_by_field_count`:::
(long) The number of _by_ field values analyzed. Note that these are counted
separately for each detector and partition.

`model_size_stats`.`total_over_field_count`:::
(long) The number of _over_ field values analyzed. Note that these are counted
separately for each detector and partition.

`model_size_stats`.`total_partition_field_count`:::
(long) The number of _partition_ field values analyzed.

`retain`::
(boolean)
include::{docdir}/ml/ml-shared.asciidoc[tag=retain]

`snapshot_id`::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=snapshot-id]

`snapshot_doc_count`::
(long) For internal use only.

`timestamp`::
(date) The creation timestamp for the snapshot.

[[ml-get-snapshot-example]]
==== {api-examples-title}

[source,console]
--------------------------------------------------
GET _ml/anomaly_detectors/farequote/model_snapshots
GET _ml/anomaly_detectors/high_sum_total_sales/model_snapshots
{
"start": "1491852977000"
"start": "1575402236000"
}
--------------------------------------------------
// TEST[skip:todo]
// TEST[skip:Kibana sample data]

In this example, the API provides a single result:
[source,js]
----
{
"count": 1,
"model_snapshots": [
"count" : 1,
"model_snapshots" : [
{
"job_id": "farequote",
"min_version": "6.3.0",
"timestamp": 1491948163000,
"description": "State persisted due to job close at 2017-04-11T15:02:43-0700",
"snapshot_id": "1491948163",
"snapshot_doc_count": 1,
"model_size_stats": {
"job_id": "farequote",
"result_type": "model_size_stats",
"model_bytes": 387594,
"total_by_field_count": 21,
"total_over_field_count": 0,
"total_partition_field_count": 20,
"bucket_allocation_failures_count": 0,
"memory_status": "ok",
"log_time": 1491948163000,
"timestamp": 1455234600000
"job_id" : "high_sum_total_sales",
"min_version" : "6.4.0",
"timestamp" : 1575402237000,
"description" : "State persisted due to job close at 2019-12-03T19:43:57+0000",
"snapshot_id" : "1575402237",
"snapshot_doc_count" : 1,
"model_size_stats" : {
"job_id" : "high_sum_total_sales",
"result_type" : "model_size_stats",
"model_bytes" : 1638816,
"model_bytes_exceeded" : 0,
"model_bytes_memory_limit" : 10485760,
"total_by_field_count" : 3,
"total_over_field_count" : 3320,
"total_partition_field_count" : 2,
"bucket_allocation_failures_count" : 0,
"memory_status" : "ok",
"log_time" : 1575402237000,
"timestamp" : 1576965600000
},
"latest_record_time_stamp": 1455235196000,
"latest_result_time_stamp": 1455234900000,
"retain": false
"latest_record_time_stamp" : 1576971072000,
"latest_result_time_stamp" : 1576965600000,
"retain" : false
}
]
}
Expand Down
39 changes: 20 additions & 19 deletions docs/reference/ml/anomaly-detection/apis/revert-snapshot.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Reverts to a specific snapshot.
[[ml-revert-snapshot-desc]]
==== {api-description-title}

The {ml} feature in {xpack} reacts quickly to anomalous input, learning new
The {ml-features} react quickly to anomalous input, learning new
behaviors in data. Highly anomalous input increases the variance in the models
whilst the system learns whether this is a new step-change in behavior or a
one-off event. In the case where this anomalous input is known to be a one-off,
Expand Down Expand Up @@ -60,8 +60,7 @@ If you want to resend data, then delete the intervening results.

[source,console]
--------------------------------------------------
POST
_ml/anomaly_detectors/high_sum_total_sales/model_snapshots/1577221697/_revert
POST _ml/anomaly_detectors/high_sum_total_sales/model_snapshots/1575402237/_revert
{
"delete_intervening_results": true
}
Expand All @@ -72,30 +71,32 @@ When the operation is complete, you receive the following results:
[source,js]
----
{
"model": {
"job_id": "high_sum_total_sales",
"min_version": "6.4.0",
"timestamp": 1577221697000,
"description": "Periodic background persist at 2019-12-24T21:08:17+0000",
"snapshot_id": "1577221697",
"snapshot_doc_count": 1,
"model_size_stats": {
"job_id": "high_sum_total_sales",
"result_type": "model_size_stats",
"model_bytes": 1325334,
"model" : {
"job_id" : "high_sum_total_sales",
"min_version" : "6.4.0",
"timestamp" : 1575402237000,
"description" : "State persisted due to job close at 2019-12-03T19:43:57+0000",
"snapshot_id" : "1575402237",
"snapshot_doc_count" : 1,
"model_size_stats" : {
"job_id" : "high_sum_total_sales",
"result_type" : "model_size_stats",
"model_bytes" : 1638816,
"model_bytes_exceeded" : 0,
"model_bytes_memory_limit" : 10485760,
"total_by_field_count" : 3,
"total_over_field_count" : 2361,
"total_over_field_count" : 3320,
"total_partition_field_count" : 2,
"bucket_allocation_failures_count" : 0,
"memory_status" : "ok",
"log_time" : 1577221697000,
"timestamp" : 1577217600000
"log_time" : 1575402237000,
"timestamp" : 1576965600000
},
"latest_record_time_stamp" : 1577221286000,
"latest_result_time_stamp" : 1577217600000,
"latest_record_time_stamp" : 1576971072000,
"latest_result_time_stamp" : 1576965600000,
"retain" : false
}
}
----

For a description of these properties, see the <<ml-get-snapshot-results,get model snapshots API>>.
104 changes: 0 additions & 104 deletions docs/reference/ml/anomaly-detection/apis/snapshotresource.asciidoc

This file was deleted.

Loading