From b3816d531ed099bfbd9ab5efb6526f404637ff1f Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 19 Jun 2019 14:02:59 -0700 Subject: [PATCH 1/7] [DOCS] Edits the get rollup capabilities API --- docs/reference/rest-api/defs.asciidoc | 2 + .../rollup/apis/jobresource.asciidoc | 31 +++++++ .../rollup/apis/rollup-caps.asciidoc | 85 ++++++++++++------- 3 files changed, 86 insertions(+), 32 deletions(-) create mode 100644 docs/reference/rollup/apis/jobresource.asciidoc diff --git a/docs/reference/rest-api/defs.asciidoc b/docs/reference/rest-api/defs.asciidoc index 65a0384a3d3bf..836b44e5ba833 100644 --- a/docs/reference/rest-api/defs.asciidoc +++ b/docs/reference/rest-api/defs.asciidoc @@ -15,6 +15,7 @@ These resource definitions are used in APIs related to {ml-features} and * <> * <> * <> +* <> * <> include::{es-repo-dir}/ml/apis/calendarresource.asciidoc[] @@ -25,5 +26,6 @@ include::{es-repo-dir}/ml/apis/jobcounts.asciidoc[] include::{es-repo-dir}/ml/apis/snapshotresource.asciidoc[] include::{es-repo-dir}/data-frames/apis/pivotresource.asciidoc[] include::{xes-repo-dir}/rest-api/security/role-mapping-resources.asciidoc[] +include::{es-repo-dir}/rollup/apis/jobresource.asciidoc[] include::{es-repo-dir}/ml/apis/resultsresource.asciidoc[] include::{es-repo-dir}/ml/apis/eventresource.asciidoc[] diff --git a/docs/reference/rollup/apis/jobresource.asciidoc b/docs/reference/rollup/apis/jobresource.asciidoc new file mode 100644 index 0000000000000..276a0c9dd24b4 --- /dev/null +++ b/docs/reference/rollup/apis/jobresource.asciidoc @@ -0,0 +1,31 @@ +[role="xpack"] +[testenv="basic"] +[[rollup-job-resource]] +=== Rollup job resources + +A rollup job resource has the following properties: + +`cron`:: + TBD + +`fields`:: + (object) TBD + +`groups`:: + TBD + +`index_pattern`:: + TBD + +`job_id`:: + (string) The unique identifier for the job. + +`metrics`:: + TBD + +`page_size`:: + TBD + +`rollup_index`:: + TBD + diff --git a/docs/reference/rollup/apis/rollup-caps.asciidoc b/docs/reference/rollup/apis/rollup-caps.asciidoc index e50806f3c1e0e..d5890ee0a591a 100644 --- a/docs/reference/rollup/apis/rollup-caps.asciidoc +++ b/docs/reference/rollup/apis/rollup-caps.asciidoc @@ -11,6 +11,22 @@ experimental[] This API returns the capabilities of any rollup jobs that have been configured for a specific index or index pattern. +[[get-rollup-caps-request]] +==== Request + +`GET _rollup/data/` + +[[get-rollup-caps-prereqs]] +==== Prerequisites + +If the {es} {security-features} are enabled, you must have `monitor`, +`monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API. +For more information, see +{stack-ov}/security-privileges.html[Security privileges]. + +[[get-rollup-caps-desc]] +===== Description + This API is useful because a rollup job is often configured to rollup only a subset of fields from the source index. Furthermore, only certain aggregations can be configured for various fields, leading to a limited subset of @@ -22,33 +38,30 @@ This API enables you to inspect an index and determine: 2. If yes to the first question, what fields were rolled up, what aggregations can be performed, and where does the data live? -==== Request - -`GET _rollup/data/{index}` - -//===== Description -==== Path Parameters +[[get-rollup-caps-params]] +==== Path parameters `index`:: - (string) Index, indices or index-pattern to return rollup capabilities for. `_all` may be used to fetch - rollup capabilities from all jobs + (string) Index, indices or index-pattern to return rollup capabilities for. + To fetch rollup capabilities from all jobs, use `_all`. + +[[get-rollup-caps-response-body]] +==== Response body +The API returns the following information: -==== Request Body +`rollup_jobs`:: + (array) An array of rollup job resources. For more information, see + <>. -There is no request body for the Get Rollup Caps API. - -==== Authorization - -You must have `monitor`, `monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API. -For more information, see -{xpack-ref}/security-privileges.html[Security Privileges]. +[[get-rollup-caps-examples]] ==== Examples -Imagine we have an index named `sensor-1` full of raw data. We know that the data will grow over time, so there -will be a `sensor-2`, `sensor-3`, etc. Let's create a Rollup job that targets the index pattern `sensor-*` to accommodate +Imagine we have an index named `sensor-1` full of raw data. We know that the +data will grow over time, so there will be a `sensor-2`, `sensor-3`, etc. Let's +create a rollup job that targets the index pattern `sensor-*` to accommodate this future scaling: [source,js] @@ -84,7 +97,8 @@ PUT _rollup/job/sensor // CONSOLE // TEST[setup:sensor_index] -We can then retrieve the rollup capabilities of that index pattern (`sensor-*`) via the following command: +We can then retrieve the rollup capabilities of that index pattern (`sensor-*`) +via the following command: [source,js] -------------------------------------------------- @@ -142,17 +156,21 @@ Which will yield the following response: ---- // TESTRESPONSE -The response that is returned contains information that is similar to the original Rollup configuration, but formatted -differently. First, there are some house-keeping details: the Rollup job's ID, the index that holds the rolled data, +The response that is returned contains information that is similar to the +original rollup configuration, but formatted differently. First, there are some +house-keeping details: the Rollup job's ID, the index that holds the rolled data, the index pattern that the job was targeting. -Next it shows a list of fields that contain data eligible for rollup searches. Here we see four fields: `node`, `temperature`, -`timestamp` and `voltage`. Each of these fields list the aggregations that are possible. For example, you can use a min, max -or sum aggregation on the `temperature` field, but only a `date_histogram` on `timestamp`. +Next it shows a list of fields that contain data eligible for rollup searches. +Here we see four fields: `node`, `temperature`, `timestamp` and `voltage`. Each +of these fields list the aggregations that are possible. For example, you can +use a min, max or sum aggregation on the `temperature` field, but only a +`date_histogram` on `timestamp`. -Note that the `rollup_jobs` element is an array; there can be multiple, independent jobs configured for a single index -or index pattern. Each of these jobs may have different configurations, so the API returns a list of all the various -configurations available. +Note that the `rollup_jobs` element is an array; there can be multiple, +independent jobs configured for a single index or index pattern. Each of these +jobs may have different configurations, so the API returns a list of all the +various configurations available. We could also retrieve the same information with a request to `_all`: @@ -163,7 +181,8 @@ GET _rollup/data/_all // CONSOLE // TEST[continued] -But note that if we use the concrete index name (`sensor-1`), we'll retrieve no rollup capabilities: +But note that if we use the concrete index name (`sensor-1`), we'll retrieve no +rollup capabilities: [source,js] -------------------------------------------------- @@ -180,7 +199,9 @@ GET _rollup/data/sensor-1 ---- // TESTRESPONSE -Why is this? The original rollup job was configured against a specific index pattern (`sensor-*`) not a concrete index -(`sensor-1`). So while the index belongs to the pattern, the rollup job is only valid across the entirety of the pattern -not just one of it's containing indices. So for that reason, the Rollup Capabilities API only returns information based -on the originally configured index name or pattern. +Why is this? The original rollup job was configured against a specific index +pattern (`sensor-*`) not a concrete index (`sensor-1`). So while the index +belongs to the pattern, the rollup job is only valid across the entirety of the +pattern not just one of its containing indices. So for that reason, the get +rollup capabilities API returns only information that is based on the originally +configured index name or pattern. From c071bc337627e3c2a0309b6c4ce899a13fbdd525 Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 19 Jun 2019 15:45:23 -0700 Subject: [PATCH 2/7] [DOCS] More edits to rollup definitions --- docs/reference/rest-api/defs.asciidoc | 2 +- .../rollup/apis/jobresource.asciidoc | 60 +++++++++++++++---- .../rollup/apis/rollup-caps.asciidoc | 20 ++++--- 3 files changed, 60 insertions(+), 22 deletions(-) diff --git a/docs/reference/rest-api/defs.asciidoc b/docs/reference/rest-api/defs.asciidoc index 836b44e5ba833..615b71fd69032 100644 --- a/docs/reference/rest-api/defs.asciidoc +++ b/docs/reference/rest-api/defs.asciidoc @@ -15,7 +15,7 @@ These resource definitions are used in APIs related to {ml-features} and * <> * <> * <> -* <> +* <> * <> include::{es-repo-dir}/ml/apis/calendarresource.asciidoc[] diff --git a/docs/reference/rollup/apis/jobresource.asciidoc b/docs/reference/rollup/apis/jobresource.asciidoc index 276a0c9dd24b4..8f31cacda4809 100644 --- a/docs/reference/rollup/apis/jobresource.asciidoc +++ b/docs/reference/rollup/apis/jobresource.asciidoc @@ -1,31 +1,65 @@ [role="xpack"] [testenv="basic"] -[[rollup-job-resource]] -=== Rollup job resources +[[rollup-capabilities]] +=== Rollup job capabilities API resources +++++ +Rollup job capabilities +++++ -A rollup job resource has the following properties: +This resource contains properties that you receive when you request information +about the capabilities of rollup jobs on specific indices or index patterns. For +more information, see <>. + +[float] +[[rollup-capabilities-properties]] +==== Properties `cron`:: - TBD - + (string) A cron string that defines when the rollup job runs. It defines an + interval of when to run the job's indexer. When the interval triggers, the + indexer attempts to roll up the data in the index pattern. The cron pattern is + unrelated to the time interval of the data being rolled up. For example, you + can create hourly rollups of your document (as defined in the + <>) but run the indexer only on a + daily basis at midnight, as defined by the cron. The cron pattern is defined + just like the {watcher} cron schedule. + `fields`:: - (object) TBD + (object) A list of fields that contain data that is eligible for rollup + searches. `groups`:: - TBD + (object) A list of the grouping fields. `index_pattern`:: - TBD + (string) The index or index pattern that is rolled up. Supports wildcard-style + patterns (for example, `logstash-*`). ++ +-- +[NOTE] +This property value cannot be a pattern that also matches the `rollup_index`. +For example, the pattern `"foo-*"` matches the rollup index `"foo-rollup"`. This +situation causes problems because the rollup job attempts to roll up its own +data at runtime. If you attempt to configure a pattern that matches the +`rollup_index`, an exception is thrown to prevent this behavior. + +-- `job_id`:: - (string) The unique identifier for the job. + (string) The unique identifier for the data that is associated with the rollup + job. It can be any alphanumeric string. `metrics`:: - TBD + (object) The metrics to collect for each group. `page_size`:: - TBD + (int) The number of bucket results that are processed on each iteration of the + rollup indexer. A larger value tends to execute faster but requires more + memory during processing. This property has no effect on how the data is + rolled up; it is merely used for tweaking the speed or memory cost of the + indexer. `rollup_index`:: - TBD - + (string) The index that contains rollup results. This index stores all the + rollup data that is generated by the job. It can be shared by multiple rollup + jobs. diff --git a/docs/reference/rollup/apis/rollup-caps.asciidoc b/docs/reference/rollup/apis/rollup-caps.asciidoc index d5890ee0a591a..2f194866310f8 100644 --- a/docs/reference/rollup/apis/rollup-caps.asciidoc +++ b/docs/reference/rollup/apis/rollup-caps.asciidoc @@ -6,16 +6,18 @@ Get rollup caps ++++ -experimental[] +Returns the capabilities of any rollup jobs that have been configured for a +specific index or index pattern. -This API returns the capabilities of any rollup jobs that have been configured -for a specific index or index pattern. +experimental[] +[discrete] [[get-rollup-caps-request]] ==== Request `GET _rollup/data/` +[discrete] [[get-rollup-caps-prereqs]] ==== Prerequisites @@ -24,6 +26,7 @@ If the {es} {security-features} are enabled, you must have `monitor`, For more information, see {stack-ov}/security-privileges.html[Security privileges]. +[discrete] [[get-rollup-caps-desc]] ===== Description @@ -38,7 +41,7 @@ This API enables you to inspect an index and determine: 2. If yes to the first question, what fields were rolled up, what aggregations can be performed, and where does the data live? - +[discrete] [[get-rollup-caps-params]] ==== Path parameters @@ -46,6 +49,7 @@ can be performed, and where does the data live? (string) Index, indices or index-pattern to return rollup capabilities for. To fetch rollup capabilities from all jobs, use `_all`. +[discrete] [[get-rollup-caps-response-body]] ==== Response body @@ -53,9 +57,9 @@ The API returns the following information: `rollup_jobs`:: (array) An array of rollup job resources. For more information, see - <>. - + <>. +[discrete] [[get-rollup-caps-examples]] ==== Examples @@ -158,7 +162,7 @@ Which will yield the following response: The response that is returned contains information that is similar to the original rollup configuration, but formatted differently. First, there are some -house-keeping details: the Rollup job's ID, the index that holds the rolled data, +house-keeping details: the rollup job's ID, the index that holds the rolled data, the index pattern that the job was targeting. Next it shows a list of fields that contain data eligible for rollup searches. @@ -200,7 +204,7 @@ GET _rollup/data/sensor-1 // TESTRESPONSE Why is this? The original rollup job was configured against a specific index -pattern (`sensor-*`) not a concrete index (`sensor-1`). So while the index +pattern (`sensor-*`) not a concrete index (`sensor-1`). So while the index belongs to the pattern, the rollup job is only valid across the entirety of the pattern not just one of its containing indices. So for that reason, the get rollup capabilities API returns only information that is based on the originally From 53daaf3cc54fce36e7db6f89ad81879c9c017ab9 Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 19 Jun 2019 16:07:22 -0700 Subject: [PATCH 3/7] [DOCS] More edits --- docs/reference/rest-api/defs.asciidoc | 4 +- .../rollup/apis/jobresource.asciidoc | 65 ------------------- .../rollup/apis/rollup-caps.asciidoc | 9 +-- 3 files changed, 7 insertions(+), 71 deletions(-) delete mode 100644 docs/reference/rollup/apis/jobresource.asciidoc diff --git a/docs/reference/rest-api/defs.asciidoc b/docs/reference/rest-api/defs.asciidoc index 615b71fd69032..8d7a0adc1ed43 100644 --- a/docs/reference/rest-api/defs.asciidoc +++ b/docs/reference/rest-api/defs.asciidoc @@ -15,7 +15,7 @@ These resource definitions are used in APIs related to {ml-features} and * <> * <> * <> -* <> +* <> * <> include::{es-repo-dir}/ml/apis/calendarresource.asciidoc[] @@ -26,6 +26,6 @@ include::{es-repo-dir}/ml/apis/jobcounts.asciidoc[] include::{es-repo-dir}/ml/apis/snapshotresource.asciidoc[] include::{es-repo-dir}/data-frames/apis/pivotresource.asciidoc[] include::{xes-repo-dir}/rest-api/security/role-mapping-resources.asciidoc[] -include::{es-repo-dir}/rollup/apis/jobresource.asciidoc[] +include::{es-repo-dir}/rollup/apis/rollup-jobs-objects.asciidoc[] include::{es-repo-dir}/ml/apis/resultsresource.asciidoc[] include::{es-repo-dir}/ml/apis/eventresource.asciidoc[] diff --git a/docs/reference/rollup/apis/jobresource.asciidoc b/docs/reference/rollup/apis/jobresource.asciidoc deleted file mode 100644 index 8f31cacda4809..0000000000000 --- a/docs/reference/rollup/apis/jobresource.asciidoc +++ /dev/null @@ -1,65 +0,0 @@ -[role="xpack"] -[testenv="basic"] -[[rollup-capabilities]] -=== Rollup job capabilities API resources -++++ -Rollup job capabilities -++++ - -This resource contains properties that you receive when you request information -about the capabilities of rollup jobs on specific indices or index patterns. For -more information, see <>. - -[float] -[[rollup-capabilities-properties]] -==== Properties - -`cron`:: - (string) A cron string that defines when the rollup job runs. It defines an - interval of when to run the job's indexer. When the interval triggers, the - indexer attempts to roll up the data in the index pattern. The cron pattern is - unrelated to the time interval of the data being rolled up. For example, you - can create hourly rollups of your document (as defined in the - <>) but run the indexer only on a - daily basis at midnight, as defined by the cron. The cron pattern is defined - just like the {watcher} cron schedule. - -`fields`:: - (object) A list of fields that contain data that is eligible for rollup - searches. - -`groups`:: - (object) A list of the grouping fields. - -`index_pattern`:: - (string) The index or index pattern that is rolled up. Supports wildcard-style - patterns (for example, `logstash-*`). -+ --- -[NOTE] -This property value cannot be a pattern that also matches the `rollup_index`. -For example, the pattern `"foo-*"` matches the rollup index `"foo-rollup"`. This -situation causes problems because the rollup job attempts to roll up its own -data at runtime. If you attempt to configure a pattern that matches the -`rollup_index`, an exception is thrown to prevent this behavior. - --- - -`job_id`:: - (string) The unique identifier for the data that is associated with the rollup - job. It can be any alphanumeric string. - -`metrics`:: - (object) The metrics to collect for each group. - -`page_size`:: - (int) The number of bucket results that are processed on each iteration of the - rollup indexer. A larger value tends to execute faster but requires more - memory during processing. This property has no effect on how the data is - rolled up; it is merely used for tweaking the speed or memory cost of the - indexer. - -`rollup_index`:: - (string) The index that contains rollup results. This index stores all the - rollup data that is generated by the job. It can be shared by multiple rollup - jobs. diff --git a/docs/reference/rollup/apis/rollup-caps.asciidoc b/docs/reference/rollup/apis/rollup-caps.asciidoc index 2f194866310f8..3d6fff8fe7d40 100644 --- a/docs/reference/rollup/apis/rollup-caps.asciidoc +++ b/docs/reference/rollup/apis/rollup-caps.asciidoc @@ -28,7 +28,7 @@ For more information, see [discrete] [[get-rollup-caps-desc]] -===== Description +==== Description This API is useful because a rollup job is often configured to rollup only a subset of fields from the source index. Furthermore, only certain aggregations @@ -57,7 +57,7 @@ The API returns the following information: `rollup_jobs`:: (array) An array of rollup job resources. For more information, see - <>. + <>. [discrete] [[get-rollup-caps-examples]] @@ -185,8 +185,7 @@ GET _rollup/data/_all // CONSOLE // TEST[continued] -But note that if we use the concrete index name (`sensor-1`), we'll retrieve no -rollup capabilities: +If we use the concrete index name (`sensor-1`): [source,js] -------------------------------------------------- @@ -195,6 +194,8 @@ GET _rollup/data/sensor-1 // CONSOLE // TEST[continued] +...we retrieve no rollup capabilities: + [source,js] ---- { From 17f1e7cee5dfde60eaaf843b5878c23deb1ae57f Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 19 Jun 2019 16:17:36 -0700 Subject: [PATCH 4/7] [DOCS] Adds object page --- .../rollup/apis/rollup-jobs-objects.asciidoc | 66 +++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 docs/reference/rollup/apis/rollup-jobs-objects.asciidoc diff --git a/docs/reference/rollup/apis/rollup-jobs-objects.asciidoc b/docs/reference/rollup/apis/rollup-jobs-objects.asciidoc new file mode 100644 index 0000000000000..789038a47809e --- /dev/null +++ b/docs/reference/rollup/apis/rollup-jobs-objects.asciidoc @@ -0,0 +1,66 @@ +[role="xpack"] +[testenv="basic"] +[[rollup-jobs-objects]] +=== rollup_jobs API objects +++++ +Rollup jobs +++++ + +The <> returns +`rollup_jobs`, which is an array of objects that provide information about the +capabilities of rollup jobs on specific indices or index patterns. For more +information, see <>. + +[float] +[[rollup-jobs-objects-properties]] +==== Properties + +`cron`:: + (string) A cron string that defines when the rollup job runs. It defines an + interval of when to run the job's indexer. When the interval triggers, the + indexer attempts to roll up the data in the index pattern. The cron pattern is + unrelated to the time interval of the data being rolled up. For example, you + can create hourly rollups of your document (as defined in the + <>) but run the indexer only on a + daily basis at midnight, as defined by the cron. The cron pattern is defined + just like the {watcher} cron schedule. + +`fields`:: + (object) A list of fields that contain data that is eligible for rollup + searches. + +`groups`:: + (object) A list of the grouping fields. + +`index_pattern`:: + (string) The index or index pattern that is rolled up. Supports wildcard-style + patterns (for example, `logstash-*`). ++ +-- +[NOTE] +This property value cannot be a pattern that also matches the `rollup_index`. +For example, the pattern `"foo-*"` matches the rollup index `"foo-rollup"`. This +situation causes problems because the rollup job attempts to roll up its own +data at runtime. If you attempt to configure a pattern that matches the +`rollup_index`, an exception is thrown to prevent this behavior. + +-- + +`job_id`:: + (string) The unique identifier for the data that is associated with the rollup + job. It can be any alphanumeric string. + +`metrics`:: + (object) The metrics to collect for each group. + +`page_size`:: + (int) The number of bucket results that are processed on each iteration of the + rollup indexer. A larger value tends to execute faster but requires more + memory during processing. This property has no effect on how the data is + rolled up; it is merely used for tweaking the speed or memory cost of the + indexer. + +`rollup_index`:: + (string) The index that contains rollup results. This index stores all the + rollup data that is generated by the job. It can be shared by multiple rollup + jobs. From a822abae01ee7a260af01afe843dad1e9d52bd4a Mon Sep 17 00:00:00 2001 From: lcawl Date: Thu, 20 Jun 2019 12:34:27 -0700 Subject: [PATCH 5/7] [DOCS] Adds comments to all rollup APIs --- .../reference/rollup/apis/delete-job.asciidoc | 54 ++++++++++++++++++- docs/reference/rollup/apis/get-job.asciidoc | 49 +++++++++++++++++ docs/reference/rollup/apis/put-job.asciidoc | 49 +++++++++++++++++ .../rollup/apis/rollup-caps.asciidoc | 49 +++++++++++++++++ .../rollup/apis/rollup-index-caps.asciidoc | 49 +++++++++++++++++ .../rollup/apis/rollup-search.asciidoc | 49 +++++++++++++++++ docs/reference/rollup/apis/start-job.asciidoc | 49 +++++++++++++++++ docs/reference/rollup/apis/stop-job.asciidoc | 48 +++++++++++++++++ 8 files changed, 394 insertions(+), 2 deletions(-) diff --git a/docs/reference/rollup/apis/delete-job.asciidoc b/docs/reference/rollup/apis/delete-job.asciidoc index 18c353ac73674..ce500238393f5 100644 --- a/docs/reference/rollup/apis/delete-job.asciidoc +++ b/docs/reference/rollup/apis/delete-job.asciidoc @@ -6,9 +6,59 @@ Delete job ++++ -experimental[] +Deletes an existing rollup job. -This API deletes an existing rollup job. A job must be *stopped* first before it can be deleted. Attempting to delete +experimental[] +//// +[float] +[[sample-api-request]] +==== {api-request-title} +// This section show the basic endpoint, without the body or optional parameters. +// Variables should use <...> syntax +// If an API supports both PUT and POST, include both here + +`PUT //_ccr/follow` + +[float] +[[sample-api-prereqs]] +==== {api-prereq-title} +// Optional list of prerequisites. + +[float] +[[sample-api-desc]] +==== {api-description-title} +// Add a more detailed description the context. +// Link to related APIs if appropriate. + +[float] +[[sample-api-path-params]] +==== {api-path-parms-title} +// A list of all path parameters in the endpoint request + +[float] +[[sample-api-query-params]] +==== {api-query-parms-title} +// A list of optional query parameters + +[float] +[[sample-api-request-body]] +==== {api-request-body-title} +// A list of the properties you can specify in the body of the request + +[float] +[[sample-api-response-body]] +==== {api-response-body-title} +// Response body is only required for detailed responses. + +[float] +[[sample-api-example]] +==== {api-example-title} +// Optional brief example. + +//// + + +A job must be *stopped* first before it can be deleted. Attempting to delete a started job will result in an error. Similarly, attempting to delete a nonexistent job will throw an exception. .Deleting the job does not delete rolled up data diff --git a/docs/reference/rollup/apis/get-job.asciidoc b/docs/reference/rollup/apis/get-job.asciidoc index 27410b8d39b7d..78acd9a171eac 100644 --- a/docs/reference/rollup/apis/get-job.asciidoc +++ b/docs/reference/rollup/apis/get-job.asciidoc @@ -6,6 +6,55 @@ Get job ++++ +//// +[float] +[[sample-api-request]] +==== {api-request-title} +// This section show the basic endpoint, without the body or optional parameters. +// Variables should use <...> syntax +// If an API supports both PUT and POST, include both here + +`PUT //_ccr/follow` + +[float] +[[sample-api-prereqs]] +==== {api-prereq-title} +// Optional list of prerequisites. + +[float] +[[sample-api-desc]] +==== {api-description-title} +// Add a more detailed description the context. +// Link to related APIs if appropriate. + +[float] +[[sample-api-path-params]] +==== {api-path-parms-title} +// A list of all path parameters in the endpoint request + +[float] +[[sample-api-query-params]] +==== {api-query-parms-title} +// A list of optional query parameters + +[float] +[[sample-api-request-body]] +==== {api-request-body-title} +// A list of the properties you can specify in the body of the request + +[float] +[[sample-api-response-body]] +==== {api-response-body-title} +// Response body is only required for detailed responses. + +[float] +[[sample-api-example]] +==== {api-example-title} +// Optional brief example. + +//// + + experimental[] This API returns the configuration, stats and status of rollup jobs. The API can return the details for a single job, diff --git a/docs/reference/rollup/apis/put-job.asciidoc b/docs/reference/rollup/apis/put-job.asciidoc index eac71a48b4336..02ff897f9ced2 100644 --- a/docs/reference/rollup/apis/put-job.asciidoc +++ b/docs/reference/rollup/apis/put-job.asciidoc @@ -6,6 +6,55 @@ Create job ++++ +//// +[float] +[[sample-api-request]] +==== {api-request-title} +// This section show the basic endpoint, without the body or optional parameters. +// Variables should use <...> syntax +// If an API supports both PUT and POST, include both here + +`PUT //_ccr/follow` + +[float] +[[sample-api-prereqs]] +==== {api-prereq-title} +// Optional list of prerequisites. + +[float] +[[sample-api-desc]] +==== {api-description-title} +// Add a more detailed description the context. +// Link to related APIs if appropriate. + +[float] +[[sample-api-path-params]] +==== {api-path-parms-title} +// A list of all path parameters in the endpoint request + +[float] +[[sample-api-query-params]] +==== {api-query-parms-title} +// A list of optional query parameters + +[float] +[[sample-api-request-body]] +==== {api-request-body-title} +// A list of the properties you can specify in the body of the request + +[float] +[[sample-api-response-body]] +==== {api-response-body-title} +// Response body is only required for detailed responses. + +[float] +[[sample-api-example]] +==== {api-example-title} +// Optional brief example. + +//// + + experimental[] This API enables you to create a rollup job. The job will be created in a `STOPPED` state, and must be diff --git a/docs/reference/rollup/apis/rollup-caps.asciidoc b/docs/reference/rollup/apis/rollup-caps.asciidoc index 3d6fff8fe7d40..3fe1bccd6981a 100644 --- a/docs/reference/rollup/apis/rollup-caps.asciidoc +++ b/docs/reference/rollup/apis/rollup-caps.asciidoc @@ -6,6 +6,55 @@ Get rollup caps ++++ +//// +[float] +[[sample-api-request]] +==== {api-request-title} +// This section show the basic endpoint, without the body or optional parameters. +// Variables should use <...> syntax +// If an API supports both PUT and POST, include both here + +`PUT //_ccr/follow` + +[float] +[[sample-api-prereqs]] +==== {api-prereq-title} +// Optional list of prerequisites. + +[float] +[[sample-api-desc]] +==== {api-description-title} +// Add a more detailed description the context. +// Link to related APIs if appropriate. + +[float] +[[sample-api-path-params]] +==== {api-path-parms-title} +// A list of all path parameters in the endpoint request + +[float] +[[sample-api-query-params]] +==== {api-query-parms-title} +// A list of optional query parameters + +[float] +[[sample-api-request-body]] +==== {api-request-body-title} +// A list of the properties you can specify in the body of the request + +[float] +[[sample-api-response-body]] +==== {api-response-body-title} +// Response body is only required for detailed responses. + +[float] +[[sample-api-example]] +==== {api-example-title} +// Optional brief example. + +//// + + Returns the capabilities of any rollup jobs that have been configured for a specific index or index pattern. diff --git a/docs/reference/rollup/apis/rollup-index-caps.asciidoc b/docs/reference/rollup/apis/rollup-index-caps.asciidoc index a0697ba70326e..77a63bbcda716 100644 --- a/docs/reference/rollup/apis/rollup-index-caps.asciidoc +++ b/docs/reference/rollup/apis/rollup-index-caps.asciidoc @@ -6,6 +6,55 @@ Get rollup index caps ++++ +//// +[float] +[[sample-api-request]] +==== {api-request-title} +// This section show the basic endpoint, without the body or optional parameters. +// Variables should use <...> syntax +// If an API supports both PUT and POST, include both here + +`PUT //_ccr/follow` + +[float] +[[sample-api-prereqs]] +==== {api-prereq-title} +// Optional list of prerequisites. + +[float] +[[sample-api-desc]] +==== {api-description-title} +// Add a more detailed description the context. +// Link to related APIs if appropriate. + +[float] +[[sample-api-path-params]] +==== {api-path-parms-title} +// A list of all path parameters in the endpoint request + +[float] +[[sample-api-query-params]] +==== {api-query-parms-title} +// A list of optional query parameters + +[float] +[[sample-api-request-body]] +==== {api-request-body-title} +// A list of the properties you can specify in the body of the request + +[float] +[[sample-api-response-body]] +==== {api-response-body-title} +// Response body is only required for detailed responses. + +[float] +[[sample-api-example]] +==== {api-example-title} +// Optional brief example. + +//// + + experimental[] This API returns the rollup capabilities of all jobs inside of a rollup index (e.g. the index where rollup data is stored). diff --git a/docs/reference/rollup/apis/rollup-search.asciidoc b/docs/reference/rollup/apis/rollup-search.asciidoc index ec2a554d09ff4..21768eed9959d 100644 --- a/docs/reference/rollup/apis/rollup-search.asciidoc +++ b/docs/reference/rollup/apis/rollup-search.asciidoc @@ -6,6 +6,55 @@ Rollup search ++++ +//// +[float] +[[sample-api-request]] +==== {api-request-title} +// This section show the basic endpoint, without the body or optional parameters. +// Variables should use <...> syntax +// If an API supports both PUT and POST, include both here + +`PUT //_ccr/follow` + +[float] +[[sample-api-prereqs]] +==== {api-prereq-title} +// Optional list of prerequisites. + +[float] +[[sample-api-desc]] +==== {api-description-title} +// Add a more detailed description the context. +// Link to related APIs if appropriate. + +[float] +[[sample-api-path-params]] +==== {api-path-parms-title} +// A list of all path parameters in the endpoint request + +[float] +[[sample-api-query-params]] +==== {api-query-parms-title} +// A list of optional query parameters + +[float] +[[sample-api-request-body]] +==== {api-request-body-title} +// A list of the properties you can specify in the body of the request + +[float] +[[sample-api-response-body]] +==== {api-response-body-title} +// Response body is only required for detailed responses. + +[float] +[[sample-api-example]] +==== {api-example-title} +// Optional brief example. + +//// + + experimental[] The Rollup Search endpoint allows searching rolled-up data using the standard query DSL. The Rollup Search endpoint diff --git a/docs/reference/rollup/apis/start-job.asciidoc b/docs/reference/rollup/apis/start-job.asciidoc index 241d070a670a0..aec0f64d0a349 100644 --- a/docs/reference/rollup/apis/start-job.asciidoc +++ b/docs/reference/rollup/apis/start-job.asciidoc @@ -6,6 +6,55 @@ Start job ++++ +//// +[float] +[[sample-api-request]] +==== {api-request-title} +// This section show the basic endpoint, without the body or optional parameters. +// Variables should use <...> syntax +// If an API supports both PUT and POST, include both here + +`PUT //_ccr/follow` + +[float] +[[sample-api-prereqs]] +==== {api-prereq-title} +// Optional list of prerequisites. + +[float] +[[sample-api-desc]] +==== {api-description-title} +// Add a more detailed description the context. +// Link to related APIs if appropriate. + +[float] +[[sample-api-path-params]] +==== {api-path-parms-title} +// A list of all path parameters in the endpoint request + +[float] +[[sample-api-query-params]] +==== {api-query-parms-title} +// A list of optional query parameters + +[float] +[[sample-api-request-body]] +==== {api-request-body-title} +// A list of the properties you can specify in the body of the request + +[float] +[[sample-api-response-body]] +==== {api-response-body-title} +// Response body is only required for detailed responses. + +[float] +[[sample-api-example]] +==== {api-example-title} +// Optional brief example. + +//// + + experimental[] This API starts an existing, stopped rollup job. If the job does not exist an exception will be thrown. diff --git a/docs/reference/rollup/apis/stop-job.asciidoc b/docs/reference/rollup/apis/stop-job.asciidoc index 35162246a5fbb..6e50ceac872bb 100644 --- a/docs/reference/rollup/apis/stop-job.asciidoc +++ b/docs/reference/rollup/apis/stop-job.asciidoc @@ -6,6 +6,54 @@ Stop job ++++ +//// +[float] +[[sample-api-request]] +==== {api-request-title} +// This section show the basic endpoint, without the body or optional parameters. +// Variables should use <...> syntax +// If an API supports both PUT and POST, include both here + +`PUT //_ccr/follow` + +[float] +[[sample-api-prereqs]] +==== {api-prereq-title} +// Optional list of prerequisites. + +[float] +[[sample-api-desc]] +==== {api-description-title} +// Add a more detailed description the context. +// Link to related APIs if appropriate. + +[float] +[[sample-api-path-params]] +==== {api-path-parms-title} +// A list of all path parameters in the endpoint request + +[float] +[[sample-api-query-params]] +==== {api-query-parms-title} +// A list of optional query parameters + +[float] +[[sample-api-request-body]] +==== {api-request-body-title} +// A list of the properties you can specify in the body of the request + +[float] +[[sample-api-response-body]] +==== {api-response-body-title} +// Response body is only required for detailed responses. + +[float] +[[sample-api-example]] +==== {api-example-title} +// Optional brief example. + +//// + experimental[] This API stops an existing, started rollup job. If the job does not exist an exception will be thrown. From e1063af2ab81010581fe22bdc195ff3faa1f4273 Mon Sep 17 00:00:00 2001 From: lcawl Date: Thu, 20 Jun 2019 13:31:35 -0700 Subject: [PATCH 6/7] [DOCS] Adds short descriptions --- .../reference/rollup/apis/delete-job.asciidoc | 136 +++++------------- docs/reference/rollup/apis/get-job.asciidoc | 8 +- docs/reference/rollup/apis/put-job.asciidoc | 7 +- .../rollup/apis/rollup-caps.asciidoc | 10 +- .../rollup/apis/rollup-index-caps.asciidoc | 9 +- .../rollup/apis/rollup-search.asciidoc | 9 +- docs/reference/rollup/apis/start-job.asciidoc | 7 +- docs/reference/rollup/apis/stop-job.asciidoc | 7 +- 8 files changed, 63 insertions(+), 130 deletions(-) diff --git a/docs/reference/rollup/apis/delete-job.asciidoc b/docs/reference/rollup/apis/delete-job.asciidoc index ce500238393f5..843e97c628838 100644 --- a/docs/reference/rollup/apis/delete-job.asciidoc +++ b/docs/reference/rollup/apis/delete-job.asciidoc @@ -9,69 +9,42 @@ Deletes an existing rollup job. experimental[] -//// + [float] [[sample-api-request]] ==== {api-request-title} -// This section show the basic endpoint, without the body or optional parameters. -// Variables should use <...> syntax -// If an API supports both PUT and POST, include both here -`PUT //_ccr/follow` +`DELETE _rollup/job/` [float] [[sample-api-prereqs]] ==== {api-prereq-title} -// Optional list of prerequisites. + +If the {es} {security-features} are enabled, you must have `manage` or +`manage_rollup` cluster privileges to use this API. For more information, see +{stack-ov}/security-privileges.html[Security privileges]. [float] [[sample-api-desc]] ==== {api-description-title} -// Add a more detailed description the context. -// Link to related APIs if appropriate. -[float] -[[sample-api-path-params]] -==== {api-path-parms-title} -// A list of all path parameters in the endpoint request +A job must be *stopped* first before it can be deleted. If you attempt to delete +a started job, an error occurs. Similarly, if you attempt to delete a +nonexistent job, an exception occurs. -[float] -[[sample-api-query-params]] -==== {api-query-parms-title} -// A list of optional query parameters - -[float] -[[sample-api-request-body]] -==== {api-request-body-title} -// A list of the properties you can specify in the body of the request - -[float] -[[sample-api-response-body]] -==== {api-response-body-title} -// Response body is only required for detailed responses. - -[float] -[[sample-api-example]] -==== {api-example-title} -// Optional brief example. - -//// - - -A job must be *stopped* first before it can be deleted. Attempting to delete -a started job will result in an error. Similarly, attempting to delete a nonexistent job will throw an exception. - -.Deleting the job does not delete rolled up data -********************************** -When a job is deleted, that only removes the process that is actively monitoring and rolling up data. -It does not delete any previously rolled up data. This is by design; a user may wish to roll up a static dataset. Because -the dataset is static, once it has been fully rolled up there is no need to keep the indexing Rollup job around (as there -will be no new data). So the job may be deleted, leaving behind the rolled up data for analysis. - -If you wish to also remove the rollup data, and the rollup index only contains the data for a single job, you can simply -delete the whole rollup index. If the rollup index stores data from several jobs, you must issue a Delete-By-Query that -targets the Rollup job's ID in the rollup index: +[IMPORTANT] +=============================== +When a job is deleted, that only removes the process that is actively monitoring +and rolling up data. It does not delete any previously rolled up data. This is +by design; a user may wish to roll up a static dataset. Because the dataset is +static, once it has been fully rolled up there is no need to keep the indexing +rollup job around (as there will be no new data). So the job can be deleted, +leaving behind the rolled up data for analysis. +If you wish to also remove the rollup data, and the rollup index only contains +the data for a single job, you can simply delete the whole rollup index. If the +rollup index stores data from several jobs, you must issue a delete-by-query +that targets the rollup job's ID in the rollup index. [source,js] -------------------------------------------------- @@ -85,34 +58,20 @@ POST my_rollup_index/_delete_by_query } -------------------------------------------------- // NOTCONSOLE +=============================== -********************************** -==== Request - -`DELETE _rollup/job/` - -//===== Description - -==== Path Parameters - -`job_id` (required):: - (string) Identifier for the job - - -==== Request Body - -There is no request body for the Delete Job API. - -==== Authorization - -You must have `manage` or `manage_rollup` cluster privileges to use this API. -For more information, see -{xpack-ref}/security-privileges.html[Security Privileges]. +[float] +[[sample-api-path-params]] +==== {api-path-parms-title} +`` (Required):: + (string) Identifier for the job. -==== Examples +[float] +[[sample-api-example]] +==== {api-example-title} -If we have a rollup job named `sensor`, it can be deleted with: +If you have a rollup job named `sensor`, you can delete it as follows: [source,js] -------------------------------------------------- @@ -121,7 +80,7 @@ DELETE _rollup/job/sensor // CONSOLE // TEST[setup:sensor_rollup_job] -Which will return the response: +The API returns the following response: [source,js] ---- @@ -130,34 +89,3 @@ Which will return the response: } ---- // TESTRESPONSE - -If however we try to delete a job which doesn't exist: - -[source,js] --------------------------------------------------- -DELETE _rollup/job/does_not_exist --------------------------------------------------- -// CONSOLE -// TEST[catch:missing] - -A 404 `resource_not_found` exception will be thrown: - -[source,js] ----- -{ - "error" : { - "root_cause" : [ - { - "type" : "resource_not_found_exception", - "reason" : "the task with id [does_not_exist] doesn't exist", - "stack_trace": ... - } - ], - "type" : "resource_not_found_exception", - "reason" : "the task with id [does_not_exist] doesn't exist", - "stack_trace": ... - }, - "status": 404 -} ----- -// TESTRESPONSE[s/"stack_trace": .../"stack_trace": $body.$_path/] diff --git a/docs/reference/rollup/apis/get-job.asciidoc b/docs/reference/rollup/apis/get-job.asciidoc index 78acd9a171eac..a8e257267e240 100644 --- a/docs/reference/rollup/apis/get-job.asciidoc +++ b/docs/reference/rollup/apis/get-job.asciidoc @@ -6,6 +6,10 @@ Get job ++++ +Returns the configuration, stats and status of rollup jobs. + +experimental[] + //// [float] [[sample-api-request]] @@ -55,9 +59,7 @@ //// -experimental[] - -This API returns the configuration, stats and status of rollup jobs. The API can return the details for a single job, +The API can return the details for a single job, or for all jobs. Note: This API only returns active (both `STARTED` and `STOPPED`) jobs. If a job was created, ran for a while then deleted, diff --git a/docs/reference/rollup/apis/put-job.asciidoc b/docs/reference/rollup/apis/put-job.asciidoc index 02ff897f9ced2..429d8c6959190 100644 --- a/docs/reference/rollup/apis/put-job.asciidoc +++ b/docs/reference/rollup/apis/put-job.asciidoc @@ -6,6 +6,9 @@ Create job ++++ +Creates a rollup job. + +experimental[] //// [float] [[sample-api-request]] @@ -55,9 +58,7 @@ //// -experimental[] - -This API enables you to create a rollup job. The job will be created in a `STOPPED` state, and must be +The job will be created in a `STOPPED` state, and must be started with the <>. ==== Request diff --git a/docs/reference/rollup/apis/rollup-caps.asciidoc b/docs/reference/rollup/apis/rollup-caps.asciidoc index 3fe1bccd6981a..b03023c6fcfff 100644 --- a/docs/reference/rollup/apis/rollup-caps.asciidoc +++ b/docs/reference/rollup/apis/rollup-caps.asciidoc @@ -6,6 +6,10 @@ Get rollup caps ++++ +Returns the capabilities of any rollup jobs that have been configured for a +specific index or index pattern. + +experimental[] //// [float] [[sample-api-request]] @@ -54,12 +58,6 @@ //// - -Returns the capabilities of any rollup jobs that have been configured for a -specific index or index pattern. - -experimental[] - [discrete] [[get-rollup-caps-request]] ==== Request diff --git a/docs/reference/rollup/apis/rollup-index-caps.asciidoc b/docs/reference/rollup/apis/rollup-index-caps.asciidoc index 77a63bbcda716..fd299abe0178a 100644 --- a/docs/reference/rollup/apis/rollup-index-caps.asciidoc +++ b/docs/reference/rollup/apis/rollup-index-caps.asciidoc @@ -6,6 +6,11 @@ Get rollup index caps ++++ +Returns the rollup capabilities of all jobs inside of a rollup index (e.g. the +index where rollup data is stored). + +experimental[] + //// [float] [[sample-api-request]] @@ -54,10 +59,6 @@ //// - -experimental[] - -This API returns the rollup capabilities of all jobs inside of a rollup index (e.g. the index where rollup data is stored). A single rollup index may store the data for multiple rollup jobs, and may have a variety of capabilities depending on those jobs. This API will allow you to determine: diff --git a/docs/reference/rollup/apis/rollup-search.asciidoc b/docs/reference/rollup/apis/rollup-search.asciidoc index 21768eed9959d..b5a6126804416 100644 --- a/docs/reference/rollup/apis/rollup-search.asciidoc +++ b/docs/reference/rollup/apis/rollup-search.asciidoc @@ -6,6 +6,10 @@ Rollup search ++++ +Searches rolled-up data using the standard query DSL. + +experimental[] + //// [float] [[sample-api-request]] @@ -54,10 +58,7 @@ //// - -experimental[] - -The Rollup Search endpoint allows searching rolled-up data using the standard query DSL. The Rollup Search endpoint +The Rollup Search endpoint is needed because, internally, rolled-up documents utilize a different document structure than the original data. The Rollup Search endpoint rewrites standard query DSL into a format that matches the rollup documents, then takes the response and rewrites it back to what a client would expect given the original query. diff --git a/docs/reference/rollup/apis/start-job.asciidoc b/docs/reference/rollup/apis/start-job.asciidoc index aec0f64d0a349..06118faa91d53 100644 --- a/docs/reference/rollup/apis/start-job.asciidoc +++ b/docs/reference/rollup/apis/start-job.asciidoc @@ -6,6 +6,9 @@ Start job ++++ +Starts an existing, stopped rollup job. + +experimental[] //// [float] [[sample-api-request]] @@ -55,9 +58,7 @@ //// -experimental[] - -This API starts an existing, stopped rollup job. If the job does not exist an exception will be thrown. + If the job does not exist an exception will be thrown. Starting an already started job has no action. ==== Request diff --git a/docs/reference/rollup/apis/stop-job.asciidoc b/docs/reference/rollup/apis/stop-job.asciidoc index 6e50ceac872bb..59a33059f695c 100644 --- a/docs/reference/rollup/apis/stop-job.asciidoc +++ b/docs/reference/rollup/apis/stop-job.asciidoc @@ -6,6 +6,9 @@ Stop job ++++ +Stops an existing, started rollup job. + +experimental[] //// [float] [[sample-api-request]] @@ -54,9 +57,7 @@ //// -experimental[] - -This API stops an existing, started rollup job. If the job does not exist an exception will be thrown. + If the job does not exist an exception will be thrown. Stopping an already stopped job has no action. ==== Request From d1c8dcfed19d61074f92b431a73e127dd73e2a17 Mon Sep 17 00:00:00 2001 From: lcawl Date: Thu, 20 Jun 2019 16:17:15 -0700 Subject: [PATCH 7/7] [DOCS] More edits --- docs/reference/rest-api/defs.asciidoc | 2 +- .../reference/rollup/apis/delete-job.asciidoc | 10 +- docs/reference/rollup/apis/get-job.asciidoc | 283 +++--------------- docs/reference/rollup/apis/put-job.asciidoc | 100 +++---- .../rollup/apis/rollup-caps.asciidoc | 2 +- .../apis/rollup-job-config-objects.asciidoc | 114 +++++++ .../apis/rollup-job-stats-objects.asciidoc | 62 ++++ .../rollup/apis/rollup-jobs-objects.asciidoc | 65 +--- 8 files changed, 269 insertions(+), 369 deletions(-) create mode 100644 docs/reference/rollup/apis/rollup-job-config-objects.asciidoc create mode 100644 docs/reference/rollup/apis/rollup-job-stats-objects.asciidoc diff --git a/docs/reference/rest-api/defs.asciidoc b/docs/reference/rest-api/defs.asciidoc index 8d7a0adc1ed43..09306ce1961fb 100644 --- a/docs/reference/rest-api/defs.asciidoc +++ b/docs/reference/rest-api/defs.asciidoc @@ -15,7 +15,7 @@ These resource definitions are used in APIs related to {ml-features} and * <> * <> * <> -* <> +* <> * <> include::{es-repo-dir}/ml/apis/calendarresource.asciidoc[] diff --git a/docs/reference/rollup/apis/delete-job.asciidoc b/docs/reference/rollup/apis/delete-job.asciidoc index 843e97c628838..d08bbf43d9723 100644 --- a/docs/reference/rollup/apis/delete-job.asciidoc +++ b/docs/reference/rollup/apis/delete-job.asciidoc @@ -11,13 +11,13 @@ Deletes an existing rollup job. experimental[] [float] -[[sample-api-request]] +[[rollup-delete-job-request]] ==== {api-request-title} `DELETE _rollup/job/` [float] -[[sample-api-prereqs]] +[[rollup-delete-job-prereqs]] ==== {api-prereq-title} If the {es} {security-features} are enabled, you must have `manage` or @@ -25,7 +25,7 @@ If the {es} {security-features} are enabled, you must have `manage` or {stack-ov}/security-privileges.html[Security privileges]. [float] -[[sample-api-desc]] +[[rollup-delete-job-desc]] ==== {api-description-title} A job must be *stopped* first before it can be deleted. If you attempt to delete @@ -61,14 +61,14 @@ POST my_rollup_index/_delete_by_query =============================== [float] -[[sample-api-path-params]] +[[rollup-delete-job-path-params]] ==== {api-path-parms-title} `` (Required):: (string) Identifier for the job. [float] -[[sample-api-example]] +[[rollup-delete-job-example]] ==== {api-example-title} If you have a rollup job named `sensor`, you can delete it as follows: diff --git a/docs/reference/rollup/apis/get-job.asciidoc b/docs/reference/rollup/apis/get-job.asciidoc index a8e257267e240..d4bb9c415dbfb 100644 --- a/docs/reference/rollup/apis/get-job.asciidoc +++ b/docs/reference/rollup/apis/get-job.asciidoc @@ -10,88 +10,65 @@ Returns the configuration, stats and status of rollup jobs. experimental[] -//// [float] -[[sample-api-request]] +[[rollup-get-job-request]] ==== {api-request-title} -// This section show the basic endpoint, without the body or optional parameters. -// Variables should use <...> syntax -// If an API supports both PUT and POST, include both here -`PUT //_ccr/follow` +`GET _rollup/job/` [float] -[[sample-api-prereqs]] +[[rollup-get-job-prereqs]] ==== {api-prereq-title} -// Optional list of prerequisites. + +If the {es} {security-features} are enabled, you must have `monitor`, +`monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API. +For more information, see +{stack-ov}/security-privileges.html[Security privileges]. [float] -[[sample-api-desc]] +[[rollup-get-job-desc]] ==== {api-description-title} -// Add a more detailed description the context. -// Link to related APIs if appropriate. -[float] -[[sample-api-path-params]] -==== {api-path-parms-title} -// A list of all path parameters in the endpoint request +The API can return the details for a single job or for all jobs. -[float] -[[sample-api-query-params]] -==== {api-query-parms-title} -// A list of optional query parameters +NOTE: This API returns only active (both `STARTED` and `STOPPED`) jobs. If a job +was created, ran for a while, then deleted, this API does not return any details +about the job. For details about historical jobs, the +<> may be more useful. [float] -[[sample-api-request-body]] -==== {api-request-body-title} -// A list of the properties you can specify in the body of the request +[[rollup-get-job-path-params]] +==== {api-path-parms-title} + +`` (Optional):: + (string) Identifier for the job to retrieve. If omitted (or `_all` is used), + all jobs are returned. [float] -[[sample-api-response-body]] +[[rollup-get-job-response-body]] ==== {api-response-body-title} -// Response body is only required for detailed responses. + +`jobs`:: + An array of rollup job resources. + + `config`::: (object) The rollup job's configuration. See + <>. + + `status`::: (object) The current status of the rollup job's indexer. For + example, `"job_state" : "stopped"`. See <>. + + `stats`::: (object) Transient statistics about the rollup job, such as how + many documents have been processed and how many rollup summary docs have + been indexed. See <>. + +For more information, see <>. [float] -[[sample-api-example]] +[[rollup-get-job-example]] ==== {api-example-title} -// Optional brief example. - -//// - - -The API can return the details for a single job, -or for all jobs. - -Note: This API only returns active (both `STARTED` and `STOPPED`) jobs. If a job was created, ran for a while then deleted, -this API will not return any details about that job. - -For details about a historical job, the <> may be more useful - -==== Request - -`GET _rollup/job/` - -//===== Description - -==== Path Parameters - -`job_id`:: - (string) Identifier for the job to retrieve. If omitted (or `_all` is used) all jobs will be returned - -==== Request Body - -There is no request body for the Get Jobs API. - -==== Authorization - -You must have `monitor`, `monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API. -For more information, see -{xpack-ref}/security-privileges.html[Security Privileges]. - -==== Examples - -If we have already created a rollup job named `sensor`, the details about the job can be retrieved with: +If you have already created a rollup job named `sensor`, you can retrieve the +details about the job: [source,js] -------------------------------------------------- @@ -100,7 +77,7 @@ GET _rollup/job/sensor // CONSOLE // TEST[setup:sensor_rollup_job] -Which will yield the following response: +The API yields the following response: [source,js] ---- @@ -165,181 +142,5 @@ Which will yield the following response: ---- // TESTRESPONSE -The `jobs` array contains a single job (`id: sensor`) since we requested a single job in the endpoint's URL. The -details for this job contain three top-level parameters: `config`, `status` and `stats` - -`config` holds the rollup job's configuration, which is identical to the configuration that was supplied when creating -the job via the <>. - -The `status` object holds the current status of the rollup job's indexer. The possible values and their meanings are: - -- `stopped` means the indexer is paused and will not process data, even if it's cron interval triggers -- `started` means the indexer is running, but not actively indexing data. When the cron interval triggers, the job's -indexer will begin to process data -- `indexing` means the indexer is actively processing data and creating new rollup documents. When in this state, any -subsequent cron interval triggers will be ignored because the job is already active with the prior trigger -- `abort` a transient state, which is usually not witnessed by the user. The `abort` state is used if the task needs to -be shut down for some reason (job has been deleted, an unrecoverable error has been encountered, etc). Shortly after -the `abort` state is set, the job will remove itself from the cluster - -Finally, the `stats` object provides transient statistics about the rollup job, such as how many documents have been -processed and how many rollup summary docs have been indexed. These stats are not persisted, so if a node is restarted -these stats will be reset. - -If we add another job, we can see how multi-job responses are handled: - -[source,js] --------------------------------------------------- -PUT _rollup/job/sensor2 <1> -{ - "index_pattern": "sensor-*", - "rollup_index": "sensor_rollup", - "cron": "*/30 * * * * ?", - "page_size" :1000, - "groups" : { - "date_histogram": { - "field": "timestamp", - "fixed_interval": "1h", - "delay": "7d" - }, - "terms": { - "fields": ["node"] - } - }, - "metrics": [ - { - "field": "temperature", - "metrics": ["min", "max", "sum"] - }, - { - "field": "voltage", - "metrics": ["avg"] - } - ] -} - -GET _rollup/job/_all <2> --------------------------------------------------- -// CONSOLE -// TEST[setup:sensor_rollup_job] -<1> We create a second job with name `sensor2` -<2> Then request all jobs by using `_all` in the GetJobs API - -Which will yield the following response: - -[source,js] ----- -{ - "jobs" : [ - { - "config" : { - "id" : "sensor2", - "index_pattern" : "sensor-*", - "rollup_index" : "sensor_rollup", - "cron" : "*/30 * * * * ?", - "groups" : { - "date_histogram" : { - "fixed_interval" : "1h", - "delay": "7d", - "field": "timestamp", - "time_zone": "UTC" - }, - "terms" : { - "fields" : [ - "node" - ] - } - }, - "metrics" : [ - { - "field" : "temperature", - "metrics" : [ - "min", - "max", - "sum" - ] - }, - { - "field" : "voltage", - "metrics" : [ - "avg" - ] - } - ], - "timeout" : "20s", - "page_size" : 1000 - }, - "status" : { - "job_state" : "stopped" - }, - "stats" : { - "pages_processed" : 0, - "documents_processed" : 0, - "rollups_indexed" : 0, - "trigger_count" : 0, - "index_failures": 0, - "index_time_in_ms": 0, - "index_total": 0, - "search_failures": 0, - "search_time_in_ms": 0, - "search_total": 0 - } - }, - { - "config" : { - "id" : "sensor", - "index_pattern" : "sensor-*", - "rollup_index" : "sensor_rollup", - "cron" : "*/30 * * * * ?", - "groups" : { - "date_histogram" : { - "fixed_interval" : "1h", - "delay": "7d", - "field": "timestamp", - "time_zone": "UTC" - }, - "terms" : { - "fields" : [ - "node" - ] - } - }, - "metrics" : [ - { - "field" : "temperature", - "metrics" : [ - "min", - "max", - "sum" - ] - }, - { - "field" : "voltage", - "metrics" : [ - "avg" - ] - } - ], - "timeout" : "20s", - "page_size" : 1000 - }, - "status" : { - "job_state" : "stopped" - }, - "stats" : { - "pages_processed" : 0, - "documents_processed" : 0, - "rollups_indexed" : 0, - "trigger_count" : 0, - "index_failures": 0, - "index_time_in_ms": 0, - "index_total": 0, - "search_failures": 0, - "search_time_in_ms": 0, - "search_total": 0 - } - } - ] -} ----- -// NOTCONSOLE +The `jobs` array contains a single job (`id: sensor`) since we requested a +single job in the endpoint's URL. diff --git a/docs/reference/rollup/apis/put-job.asciidoc b/docs/reference/rollup/apis/put-job.asciidoc index 429d8c6959190..fb13598bbd40b 100644 --- a/docs/reference/rollup/apis/put-job.asciidoc +++ b/docs/reference/rollup/apis/put-job.asciidoc @@ -9,103 +9,71 @@ Creates a rollup job. experimental[] -//// + [float] [[sample-api-request]] ==== {api-request-title} -// This section show the basic endpoint, without the body or optional parameters. -// Variables should use <...> syntax -// If an API supports both PUT and POST, include both here -`PUT //_ccr/follow` +`PUT _rollup/job/` [float] [[sample-api-prereqs]] ==== {api-prereq-title} -// Optional list of prerequisites. + +If the {es} {security-features} are enabled, you must have `manage` or +`manage_rollup` cluster privileges to use this API. For more information, see +{stack-ov}/security-privileges.html[Security privileges]. [float] [[sample-api-desc]] ==== {api-description-title} -// Add a more detailed description the context. -// Link to related APIs if appropriate. + +Jobs are created in a `STOPPED` state. You can start them with the +<>. [float] [[sample-api-path-params]] ==== {api-path-parms-title} -// A list of all path parameters in the endpoint request -[float] -[[sample-api-query-params]] -==== {api-query-parms-title} -// A list of optional query parameters +`job_id` (required):: + (string) Identifier for the job. [float] [[sample-api-request-body]] ==== {api-request-body-title} -// A list of the properties you can specify in the body of the request - -[float] -[[sample-api-response-body]] -==== {api-response-body-title} -// Response body is only required for detailed responses. - -[float] -[[sample-api-example]] -==== {api-example-title} -// Optional brief example. - -//// - - -The job will be created in a `STOPPED` state, and must be -started with the <>. - -==== Request -`PUT _rollup/job/` - -//===== Description - -==== Path Parameters - -`job_id` (required):: - (string) Identifier for the job - - -==== Request Body - -`index_pattern` (required):: - (string) The index, or index pattern, that you wish to rollup. Supports wildcard-style patterns (`logstash-*`). - -`rollup_index` (required):: - (string) The index that you wish to store rollup results into. Can be shared with other rollup jobs. - -`cron` (required):: +`cron` (Required):: (string) A cron string which defines when the rollup job should be executed. -`page_size` (required):: - (int) The number of bucket results that should be processed on each iteration of the rollup indexer. A larger value - will tend to execute faster, but will require more memory during processing. +`groups` (Required):: + (object) Defines the grouping fields that are defined for this rollup job. See + <>. -`groups` (required):: - (object) Defines the grouping fields that are defined for this rollup job. See <>. +`index_pattern` (Required):: + (string) The index or index pattern to roll up. Supports wildcard-style + patterns (`logstash-*`). -`metrics`:: - (object) Defines the metrics that should be collected for each grouping tuple. See <>. - -For more details about the job configuration, see <>. +`metrics` (Optional):: + (object) Defines the metrics to collect for each grouping tuple. See + <>. -==== Authorization +`page_size` (Required):: + (integer) The number of bucket results that are processed on each iteration of + the rollup indexer. A larger value tends to execute faster, but requires more + memory during processing. -You must have `manage` or `manage_rollup` cluster privileges to use this API. -For more information, see -{xpack-ref}/security-privileges.html[Security Privileges]. +`rollup_index` (Required):: + (string) The index that contains the rollup results. The index can be shared + with other rollup jobs. +For more details about the job configuration, see <>. -==== Examples +[float] +[[sample-api-example]] +==== {api-example-title} -The following example creates a rollup job named "sensor", targeting the "sensor-*" index pattern: +The following example creates a rollup job named "sensor", targeting the +"sensor-*" index pattern: [source,js] -------------------------------------------------- diff --git a/docs/reference/rollup/apis/rollup-caps.asciidoc b/docs/reference/rollup/apis/rollup-caps.asciidoc index b03023c6fcfff..d3c09fa1b3d7c 100644 --- a/docs/reference/rollup/apis/rollup-caps.asciidoc +++ b/docs/reference/rollup/apis/rollup-caps.asciidoc @@ -104,7 +104,7 @@ The API returns the following information: `rollup_jobs`:: (array) An array of rollup job resources. For more information, see - <>. + <>. [discrete] [[get-rollup-caps-examples]] diff --git a/docs/reference/rollup/apis/rollup-job-config-objects.asciidoc b/docs/reference/rollup/apis/rollup-job-config-objects.asciidoc new file mode 100644 index 0000000000000..a583bdb372b12 --- /dev/null +++ b/docs/reference/rollup/apis/rollup-job-config-objects.asciidoc @@ -0,0 +1,114 @@ +[role="xpack"] +[testenv="basic"] +[[rollup-job-config-objects]] +=== Rollup job config API objects +++++ +Rollup job config +++++ + +The `config` object holds the rollup job's configuration. It is used by the +<> and the +<>. + +`cron`:: + (string) A cron string that defines when the rollup job executes. + +`groups`:: + (object) Defines the grouping fields that are defined for this rollup job. See + <> and <>. + +`id`:: + (string) An identifier for the rollup job. This property is informational; it + is returned by the <>. + +`index_pattern`:: + (string) The index or index pattern to roll up. It supports wildcard-style + patterns such as `logstash-*`. + +`metrics`:: + (object) Defines the metrics are collected for each grouping tuple. + See <> and <>. + +`page_size`:: + (integer) The number of bucket results that are processed on each iteration of + the rollup indexer. A larger value tends to execute faster, but requires more + memory during processing. + +`rollup_index`:: + (string) The index that contains the rollup results. The index can be shared + with other rollup jobs. + +//// +`timeout`:: TBD. Exists in get jobs but not put job. +//// + +[float] +[[rollup-job-group-objects]] +==== Rollup job group API objects + +`date_histogram`:: (object) TBD + +[float] +[[rollup-job-metrics-objects]] +==== Rollup job metrics API objects + +The `metrics` resource is an array of the following properties: + +`field`:: TBD + +metrics:: (array) TBD + +//// +`config`:: + (object) TBD + +`cron`:: + (string) A cron string that defines when the rollup job runs. It defines an + interval of when to run the job's indexer. When the interval triggers, the + indexer attempts to roll up the data in the index pattern. The cron pattern is + unrelated to the time interval of the data being rolled up. For example, you + can create hourly rollups of your document (as defined in the + <>) but run the indexer only on a + daily basis at midnight, as defined by the cron. The cron pattern is defined + just like the {watcher} cron schedule. + +`fields`:: + (object) A list of fields that contain data that is eligible for rollup + searches. + +`groups`:: + (object) A list of the grouping fields. + +`index_pattern`:: + (string) The index or index pattern that is rolled up. Supports wildcard-style + patterns (for example, `logstash-*`). ++ +-- +[NOTE] +This property value cannot be a pattern that also matches the `rollup_index`. +For example, the pattern `"foo-*"` matches the rollup index `"foo-rollup"`. This +situation causes problems because the rollup job attempts to roll up its own +data at runtime. If you attempt to configure a pattern that matches the +`rollup_index`, an exception is thrown to prevent this behavior. + +-- + +`job_id`:: + (string) The unique identifier for the data that is associated with the rollup + job. It can be any alphanumeric string. + +`metrics`:: + (object) The metrics to collect for each group. + +`page_size`:: + (int) The number of bucket results that are processed on each iteration of the + rollup indexer. A larger value tends to execute faster but requires more + memory during processing. This property has no effect on how the data is + rolled up; it is merely used for tweaking the speed or memory cost of the + indexer. + +`rollup_index`:: + (string) The index that contains rollup results. This index stores all the + rollup data that is generated by the job. It can be shared by multiple rollup + jobs. +//// diff --git a/docs/reference/rollup/apis/rollup-job-stats-objects.asciidoc b/docs/reference/rollup/apis/rollup-job-stats-objects.asciidoc new file mode 100644 index 0000000000000..f8db3889df6d0 --- /dev/null +++ b/docs/reference/rollup/apis/rollup-job-stats-objects.asciidoc @@ -0,0 +1,62 @@ +[role="xpack"] +[testenv="basic"] +[[rollup-job-stats-objects]] +=== Rollup job stats and status API objects +++++ +Rollup job stats and status +++++ + +The <> retrieves the following objects, +which provide information about the status and statistics for each job. + +[float] +[[rollup-job-stats-properties]] +==== {api-definitions-title} + +`status`:: (object) The current status of the rollup job's indexer. + `job_state`::: (string) Valid values are: ++ +-- +* `stopped`: The indexer is paused and will not process data, even if its cron +interval triggers. +* `started`: The indexer is running, but not actively indexing data. When the +cron interval triggers, the job's indexer will begin to process data. +* `indexing`: The indexer is actively processing data and creating new rollup +documents. When in this state, any subsequent cron interval triggers will be +ignored because the job is already active with the prior trigger. +* `abort`: A transient state, which is usually not witnessed by the user. The +`abort` state is used if the task needs to be shut down for some reason. For +example, the job has been deleted or an unrecoverable error has been encountered. Shortly after the `abort` state is set, the job will remove itself from the +cluster. +-- + +`stats`:: + (object) Transient statistics about the rollup job. ++ +-- +NOTE: These stats are not persisted. If a node is restarted, these stats are +reset. + +-- + + `pages_processed`::: TBD. `0`. + + `documents_processed`:: The number of documents that have been processed. For + example, `0`. + + `rollups_indexed`:: The number of rollup summary docs that have been indexed. + For example, `0`. + + `trigger_count`:: TBD. `0`. + + `index_failures`:: TBD. `0`. + + `index_time_in_ms`:: TBD. `0`. + + `index_total`:: TBD. `0`. + + `search_failures`:: TBD. `0`. + + `search_time_in_ms`:: TBD. `0`. + + `search_total`:: TBD diff --git a/docs/reference/rollup/apis/rollup-jobs-objects.asciidoc b/docs/reference/rollup/apis/rollup-jobs-objects.asciidoc index 789038a47809e..4cda6a59e8f03 100644 --- a/docs/reference/rollup/apis/rollup-jobs-objects.asciidoc +++ b/docs/reference/rollup/apis/rollup-jobs-objects.asciidoc @@ -1,66 +1,21 @@ [role="xpack"] [testenv="basic"] -[[rollup-jobs-objects]] -=== rollup_jobs API objects +[[rollup-job-objects]] +=== Rollup job API objects ++++ Rollup jobs ++++ The <> returns `rollup_jobs`, which is an array of objects that provide information about the -capabilities of rollup jobs on specific indices or index patterns. For more -information, see <>. +capabilities of rollup jobs on specific indices or index patterns. -[float] -[[rollup-jobs-objects-properties]] -==== Properties +The <> returns: -`cron`:: - (string) A cron string that defines when the rollup job runs. It defines an - interval of when to run the job's indexer. When the interval triggers, the - indexer attempts to roll up the data in the index pattern. The cron pattern is - unrelated to the time interval of the data being rolled up. For example, you - can create hourly rollups of your document (as defined in the - <>) but run the indexer only on a - daily basis at midnight, as defined by the cron. The cron pattern is defined - just like the {watcher} cron schedule. - -`fields`:: - (object) A list of fields that contain data that is eligible for rollup - searches. +`jobs`:: (array) TBD + `config`::: (object) TBD. See <>. + `status`::: (object) TBD. See <>. + `stats`::: (object) TBD. See <>. -`groups`:: - (object) A list of the grouping fields. - -`index_pattern`:: - (string) The index or index pattern that is rolled up. Supports wildcard-style - patterns (for example, `logstash-*`). -+ --- -[NOTE] -This property value cannot be a pattern that also matches the `rollup_index`. -For example, the pattern `"foo-*"` matches the rollup index `"foo-rollup"`. This -situation causes problems because the rollup job attempts to roll up its own -data at runtime. If you attempt to configure a pattern that matches the -`rollup_index`, an exception is thrown to prevent this behavior. - --- - -`job_id`:: - (string) The unique identifier for the data that is associated with the rollup - job. It can be any alphanumeric string. - -`metrics`:: - (object) The metrics to collect for each group. - -`page_size`:: - (int) The number of bucket results that are processed on each iteration of the - rollup indexer. A larger value tends to execute faster but requires more - memory during processing. This property has no effect on how the data is - rolled up; it is merely used for tweaking the speed or memory cost of the - indexer. - -`rollup_index`:: - (string) The index that contains rollup results. This index stores all the - rollup data that is generated by the job. It can be shared by multiple rollup - jobs. +include::rollup-job-config-objects.asciidoc[] +include::rollup-job-stats-objects.asciidoc[]