[DOCS] Adds retention_policy to PUT Transform API docs (#68656)

Co-authored-by: lcawl <[email protected]>

Author: szabosteve

docs/reference/rest-api/common-parms.asciidoc

@@ -200,8 +200,8 @@ history of operations on a shard.
 end::docs-deleted[]
 
 tag::docs-deleted-transform[]
-The number of documents that have been deleted from the destination index
-for the {transform}.
+The number of documents that have been deleted from the destination index due to
+the retention policy for this {transform}.
 end::docs-deleted-transform[]
 
 tag::docs-indexed[]
@@ -990,6 +990,24 @@ The `latest` method transforms the data by finding the latest document for each
 unique key.
 end::transform-latest[]
 
+tag::transform-retention[]
+Defines a retention policy for the {transform}. Data that meets the defined 
+criteria is deleted from the destination index.
+end::transform-retention[]
+
+tag::transform-retention-time[]
+Specifies that the {transform} uses a time field to set the retention policy.
+end::transform-retention-time[]
+
+tag::transform-retention-time-field[]
+The date field that is used to calculate the age of the document.
+end::transform-retention-time-field[]
+
+tag::transform-retention-time-max-age[]
+Specifies the maximum age of a document in the destination index. Documents that 
+are older than the configured value are removed from the destination index.
+end::transform-retention-time-max-age[]
+
 tag::transform-settings[]
 Defines optional {transform} settings.
 end::transform-settings[]

docs/reference/transform/apis/get-transform-stats.asciidoc

@@ -310,6 +310,8 @@ The API returns the following results:
         "pages_processed" : 78,
         "documents_processed" : 6027,
         "documents_indexed" : 68,
+        "documents_deleted": 22,
+        "delete_time_in_ms": 214,
         "trigger_count" : 168,
         "index_time_in_ms" : 412,
         "index_total" : 20,

docs/reference/transform/apis/preview-transform.asciidoc

@@ -120,6 +120,32 @@ include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=pivot-group-by]
 ====
 //End pivot
 
+//Begin retention policy
+`retention_policy`::
+(Optional, object)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention]
++
+.Properties of `retention_policy`
+[%collapsible%open]
+====
+`time`:::
+(Required, object)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time]
++
+.Properties of `time`
+[%collapsible%open]
+=====
+`field`:::
+(Required, string)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time-field]
+
+`max_age`:::
+(Required, <<time-units, time units>>)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time-max-age]
+=====
+====
+//End retention policy
+
 //Begin source
 `source`::
 (Required, object)

docs/reference/transform/apis/put-transform.asciidoc

@@ -32,16 +32,17 @@ For more information, see <<built-in-roles>>, <<security-privileges>>, and
 [[put-transform-desc]]
 == {api-description-title}
 
-This API defines a {transform}, which copies data from source indices,
+This API defines a {transform}, which copies data from source indices, 
 transforms it, and persists it into an entity-centric destination index. If you
 choose to use the pivot method for your {transform}, the entities are defined by
 the set of `group_by` fields in the `pivot` object.  If you choose to use the
 latest method, the entities are defined by the `unique_key` field values in the
 `latest` object.
 
-You can also think of the destination index as a two-dimensional tabular data structure (known as a {dataframe}). The ID for each document in the
-{dataframe} is generated from a hash of the entity, so there is a unique row
-per entity. For more information, see <<transforms>>.
+You can also think of the destination index as a two-dimensional tabular data 
+structure (known as a {dataframe}). The ID for each document in the {dataframe} 
+is generated from a hash of the entity, so there is a unique row per entity. For 
+more information, see <<transforms>>.
 
 When the {transform} is created, a series of validations occur to
 ensure its success. For example, there is a check for the existence of the
@@ -151,6 +152,32 @@ include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=pivot-group-by]
 ====
 //End pivot
 
+//Begin retention policy
+`retention_policy`::
+(Optional, object)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention]
++
+.Properties of `retention_policy`
+[%collapsible%open]
+====
+`time`:::
+(Required, object)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time]
++
+.Properties of `time`
+[%collapsible%open]
+=====
+`field`:::
+(Required, string)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time-field]
+
+`max_age`:::
+(Required, <<time-units, time units>>)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time-max-age]
+=====
+====
+//End retention policy
+
 //Begin settings
 `settings`::
 (Optional, object)
@@ -279,6 +306,12 @@ PUT _transform/ecommerce_transform1
       "field": "order_date",
       "delay": "60s"
     }
+  },
+  "retention_policy": {
+    "time": {
+      "field": "order_date",
+      "max_age": "30d"
+    }
   }
 }
 --------------------------------------------------

docs/reference/transform/apis/update-transform.asciidoc

@@ -40,8 +40,8 @@ When the {transform} is updated, a series of validations occur to ensure its
 success. You can use the `defer_validation` parameter to skip these checks.
 
 All updated properties except description do not take effect until after the
-{transform} starts the next checkpoint. This is so there is consistency with the
-pivoted data in each checkpoint.
+{transform} starts the next checkpoint. This is so there is data consistency in
+each checkpoint.
 
 [IMPORTANT]
 ====
@@ -102,6 +102,32 @@ include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=dest-pipeline]
 (Optional, <<time-units, time units>>)
 include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=frequency]
 
+//Begin retention policy
+`retention_policy`::
+(Optional, object)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention]
++
+.Properties of `retention_policy`
+[%collapsible%open]
+====
+`time`:::
+(Required, object)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time]
++
+.Properties of `time`
+[%collapsible%open]
+=====
+`field`:::
+(Required, string)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time-field]
+
+`max_age`:::
+(Required, <<time-units, time units>>)
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time-max-age]
+=====
+====
+//End retention policy
+
 //Begin settings
 `settings`::
 (Optional, object)