diff --git a/docs/reference/cat/transforms.asciidoc b/docs/reference/cat/transforms.asciidoc index 78013e394ef24..81c22a31d91b1 100644 --- a/docs/reference/cat/transforms.asciidoc +++ b/docs/reference/cat/transforms.asciidoc @@ -6,23 +6,180 @@ Returns configuration and usage information about {transforms}. - [[cat-transforms-api-request]] ==== {api-request-title} +`GET /_cat/transforms/` + + +`GET /_cat/transforms/_all` + + +`GET /_cat/transforms/*` + + `GET /_cat/transforms` +[[cat-transforms-api-prereqs]] +==== {api-prereq-title} + +* If the {es} {security-features} are enabled, you must have `monitor_transform` +cluster privileges to use this API. The built-in `transform_user` role has these +privileges. For more information, see <> and +<>. //[[cat-transforms-api-desc]] //==== {api-description-title} +[[cat-transforms-api-path-params]] +==== {api-path-parms-title} + +``:: +(Optional, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id-wildcard] + +[[cat-transforms-api-query-params]] +==== {api-query-parms-title} + +`allow_no_match`:: +(Optional, boolean) +include::{docdir}/rest-api/common-parms.asciidoc[tag=allow-no-match-transforms1] + +include::{docdir}/rest-api/common-parms.asciidoc[tag=http-format] + +`from`:: +(Optional, integer) +include::{docdir}/rest-api/common-parms.asciidoc[tag=from-transforms] + +include::{docdir}/rest-api/common-parms.asciidoc[tag=cat-h] ++ +If you do not specify which columns to include, the API returns the default +columns. If you explicitly specify one or more columns, it returns only the +specified columns. ++ +Valid columns are: + +`changes_last_detection_time`, `cldt`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=checkpointing-changes-last-detected-at] + +`checkpoint_duration_time_exp_avg`, `cdtea`, `checkpointTimeExpAvg`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-checkpoint-duration-ms] + +`create_time`, `ct`, `createTime`::: +(Default) +The time the {transform} was created. + +`description`, `d`::: +(Default) +The description of the {transform}. + +`dest_index`, `di`, `destIndex`::: +(Default) +include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-index] + +`document_total`, `dt`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=docs-processed] + +`frequency`, `f`::: +(Default) +include::{docdir}/rest-api/common-parms.asciidoc[tag=frequency] + +`id`::: +(Default) +include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id] + +`index_failure`, `if`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=index-failures] + +`index_time`, `itime`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=index-time-ms] + +`index_total`, `it`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=index-total] + +`indexed_documents_exp_avg`, `idea`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-documents-indexed] + +`invocation_total`, `itotal`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=trigger-count] + +`max_page_search_size`, `mpsz`::: +(Default) +include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-max-page-search-size] + +`page_total`, `pt`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=pages-processed] + +`pipeline`, `p`::: +(Default) +include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-pipeline] + +`processed_documents_exp_avg`, `pdea`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-documents-processed] + +`reason`, `r`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=state-transform-reason] + +`search_failure`, `sf`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=search-failures] + +`search_time`, `stime`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=search-time-ms] + +`search_total`, `st`::: +include::{docdir}/rest-api/common-parms.asciidoc[tag=search-total] + +`source_index`, `si`, `sourceIndex`::: +(Default) +include::{docdir}/rest-api/common-parms.asciidoc[tag=source-index-transforms] + +`state`, `s`::: +(Default) +include::{docdir}/rest-api/common-parms.asciidoc[tag=state-transform] + +`transform_type`, `tt`::: +(Default) +Indicates the type of {transform}: `batch` or `continuous`. + +`version`, `v`::: +(Default) +The version of {es} that existed on the node when the {transform} was +created. + +include::{docdir}/rest-api/common-parms.asciidoc[tag=help] + +include::{docdir}/rest-api/common-parms.asciidoc[tag=cat-s] + +`size`:: +(Optional, integer) +include::{docdir}/rest-api/common-parms.asciidoc[tag=size-transforms] + +include::{docdir}/rest-api/common-parms.asciidoc[tag=time] -//[[cat-transforms-api-query-params]] -//==== {api-query-parms-title} +include::{docdir}/rest-api/common-parms.asciidoc[tag=cat-v] +[[cat-transforms-api-examples]] +==== {api-examples-title} -//[[cat-transforms-api-response-codes]] -//==== {api-response-codes-title} +[source,console] +-------------------------------------------------- +GET /_cat/transforms?v&format=json +-------------------------------------------------- +// TEST[skip:kibana sample data] -//[[cat-transforms-api-examples]] -//==== {api-examples-title} +[source,console-result] +---- +[ + { + "id" : "ecommerce_transform", + "create_time" : "2020-03-20T20:31:25.077Z", + "version" : "7.7.0", + "source_index" : "kibana_sample_data_ecommerce", + "dest_index" : "kibana_sample_data_ecommerce_transform", + "pipeline" : null, + "description" : "Maximum priced ecommerce data by customer_id in Asia", + "transform_type" : "continuous", + "frequency" : "5m", + "max_page_search_size" : "500", + "state" : "STARTED" + } +] +---- +// TESTRESPONSE[skip:kibana sample data] diff --git a/docs/reference/rest-api/common-parms.asciidoc b/docs/reference/rest-api/common-parms.asciidoc index 5cc8116257c08..c52357de31aad 100644 --- a/docs/reference/rest-api/common-parms.asciidoc +++ b/docs/reference/rest-api/common-parms.asciidoc @@ -109,6 +109,10 @@ tag::bytes[] (Optional, <>) Unit used to display byte values. end::bytes[] +tag::checkpointing-changes-last-detected-at[] +The timestamp when changes were last detected in the source indices. +end::checkpointing-changes-last-detected-at[] + tag::cluster-health-status[] (string) Health status of the cluster, based on the state of its primary and replica @@ -188,6 +192,11 @@ is based on Lucene documents. {es} reclaims the disk space of deleted Lucene documents when a segment is merged. end::docs-deleted[] +tag::docs-processed[] +The number of documents that have been processed from the source index of +the {transform}. +end::docs-processed[] + tag::enrich-policy[] Enrich policy name used to limit the request. @@ -217,6 +226,20 @@ Wildcard expressions are not accepted. -- end::expand-wildcards[] +tag::exponential-avg-checkpoint-duration-ms[] +Exponential moving average of the duration of the checkpoint, in milliseconds. +end::exponential-avg-checkpoint-duration-ms[] + +tag::exponential-avg-documents-indexed[] +Exponential moving average of the number of new documents that have been +indexed. +end::exponential-avg-documents-indexed[] + +tag::exponential-avg-documents-processed[] +Exponential moving average of the number of documents that have been +processed. +end::exponential-avg-documents-processed[] + tag::field_statistics[] `field_statistics`:: (Optional, boolean) If `true`, the response includes the document count, sum of document frequencies, @@ -395,6 +418,25 @@ tag::index[] used to limit the request. end::index[] +tag::index-failures[] +The number of indexing failures. +end::index-failures[] + +tag::index-time-ms[] +The amount of time spent indexing, in milliseconds. +end::index-time-ms[] + +tag::index-total[] +The number of indices created. +end::index-total[] + +tag::bulk-index[] +`_index`:: +(Optional, string) +The name of the target index. +Required if not specified as a path parameter. +end::bulk-index[] + tag::index-metric[] ``:: + @@ -582,6 +624,11 @@ tag::pipeline[] (Optional, string) ID of the pipeline to use to preprocess incoming documents. end::pipeline[] +tag::pages-processed[] +The number of search or bulk index operations processed. Documents are +processed in batches instead of individually. +end::pages-processed[] + tag::path-pipeline[] ``:: (Optional, string) Comma-separated list or wildcard expression of pipeline IDs @@ -716,6 +763,14 @@ tag::scroll_size[] Defaults to 100. end::scroll_size[] +tag::search-failures[] +The number of search failures. +end::search-failures[] + +tag::search-time-ms[] +The amount of time spent searching, in milliseconds. +end::search-time-ms[] + tag::search_timeout[] `timeout`:: (Optional, <>) @@ -723,6 +778,10 @@ Explicit timeout for each search request. Defaults to no timeout. end::search_timeout[] +tag::search-total[] +The number of search operations on the source index for the {transform}. +end::search-total[] + tag::search_type[] `search_type`:: (Optional, string) The type of the search operation. Available options: @@ -812,6 +871,26 @@ A query clause that retrieves a subset of data from the source index. See <>. end::source-query-transforms[] +tag::state-transform[] +The status of the {transform}, which can be one of the following values: ++ +-- +* `aborting`: The {transform} is aborting. +* `failed`: The {transform} failed. For more information about the failure, +check the reason field. +* `indexing`: The {transform} is actively processing data and creating new +documents. +* `started`: The {transform} is running but not actively indexing data. +* `stopped`: The {transform} is stopped. +* `stopping`: The {transform} is stopping. +-- +end::state-transform[] + +tag::state-transform-reason[] +If a {transform} has a `failed` state, this property provides details about the +reason for the failure. +end::state-transform-reason[] + tag::stats[] `stats`:: (Optional, string) Specific `tag` of the request for logging and statistical @@ -914,6 +993,13 @@ expression. If you do not specify one of these options, the API returns information for all {transforms}. end::transform-id-wildcard[] +tag::trigger-count[] +The number of times the {transform} has been triggered by the scheduler. For +example, the scheduler triggers the {transform} indexer to check for updates +or ingest new data at an interval specified in the +<>. +end::trigger-count[] + tag::cat-v[] `v`:: (Optional, boolean) If `true`, the response includes column headings. diff --git a/docs/reference/transform/apis/get-transform-stats.asciidoc b/docs/reference/transform/apis/get-transform-stats.asciidoc index 1175fba036bb4..57bb61ebac31d 100644 --- a/docs/reference/transform/apis/get-transform-stats.asciidoc +++ b/docs/reference/transform/apis/get-transform-stats.asciidoc @@ -77,7 +77,8 @@ informational; you cannot update their values. `checkpointing`:: (object) Contains statistics about <>. `checkpointing`.`changes_last_detected_at`::: -(date) The timestamp when changes were last detected in the source indices. +(date) +include::{docdir}/rest-api/common-parms.asciidoc[tag=checkpointing-changes-last-detected-at] `checkpointing`.`last`::: (object) Contains statistics about the last completed checkpoint. `checkpointing`.`last`.`checkpoint`:::: @@ -127,56 +128,53 @@ started. `node`.`transport_address`::: (string) The host and port where transport HTTP connections are accepted. For example, `127.0.0.1:9300`. +`reason`:: +(string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=state-transform-reason] `state`:: -(string) The status of the {transform}, which can be one of the following values: -+ --- -* `aborting`: The {transform} is aborting. -* `failed`: The {transform} failed. For more information about the failure, -check the reason field. -* `indexing`: The {transform} is actively processing data and creating new -documents. -* `started`: The {transform} is running but not actively indexing data. -* `stopped`: The {transform} is stopped. -* `stopping`: The {transform} is stopping. --- - +(string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=state-transform] `stats`:: (object) An object that provides statistical information about the {transform}. `stats`.`documents_indexed`::: (long) The number of documents that have been indexed into the destination index for the {transform}. `stats`.`documents_processed`::: -(long) The number of documents that have been processed from the source index of -the {transform}. +(long) +include::{docdir}/rest-api/common-parms.asciidoc[tag=docs-processed] `stats`.`exponential_avg_checkpoint_duration_ms`::: -(double) Exponential moving average of the duration of the checkpoint, in milliseconds. +(double) +include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-checkpoint-duration-ms] `stats`.`exponential_avg_documents_indexed`::: -(double) Exponential moving average of the number of new documents that have been -indexed. +(double) +include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-documents-indexed] `stats`.`exponential_avg_documents_processed`::: -(double) Exponential moving average of the number of documents that have been -processed. +(double) +include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-documents-processed] `stats`.`index_failures`::: -(long) The number of indexing failures. +(long) +include::{docdir}/rest-api/common-parms.asciidoc[tag=index-failures] `stats`.`index_time_in_ms`::: -(long) The amount of time spent indexing, in milliseconds. +(long) +include::{docdir}/rest-api/common-parms.asciidoc[tag=index-time-ms] `stats`.`index_total`::: -(long) The number of indices created. +(long) +include::{docdir}/rest-api/common-parms.asciidoc[tag=index-total] `stats`.`pages_processed`::: -(long) The number of search or bulk index operations processed. Documents are -processed in batches instead of individually. +(long) +include::{docdir}/rest-api/common-parms.asciidoc[tag=pages-processed] `stats`.`search_failures`::: -(long) The number of search failures. +(long) +include::{docdir}/rest-api/common-parms.asciidoc[tag=search-failures] `stats`.`search_time_in_ms`::: -(long) The amount of time spent searching, in milliseconds. +(long) +include::{docdir}/rest-api/common-parms.asciidoc[tag=search-time-ms] `stats`.`search_total`::: -(long) The number of search operations on the source index for the {transform}. +(long) +include::{docdir}/rest-api/common-parms.asciidoc[tag=search-total] `stats`.`trigger_count`::: -(long) The number of times the {transform} has been triggered by the scheduler. -For example, the scheduler triggers the {transform} indexer to check for updates -or ingest new data at an interval specified in the -<>. +(long) +include::{docdir}/rest-api/common-parms.asciidoc[tag=trigger-count] [[get-transform-stats-response-codes]] ==== {api-response-codes-title}