[DOCS] Convert 'Restore a snapshot' to tutorial (#76929)

Updates the 'Restore a snapshot' guide to be more tutorial-focused.
Adds a tutorial for restoring an entire cluster.

Closes #72497.

Author: jrodewig

docs/build.gradle

@@ -1515,6 +1515,29 @@ setups['setup-repository'] = '''
             location: buildDir/cluster/shared/repo
 '''
 
+// Used by snapshot restore docs
+setups['setup-snapshots'] = setups['setup-repository'] + '''
+  - do:
+      bulk:
+        index: my-index
+        refresh: true
+        body: |
+          {"index":{"_id": "0"}}
+          {"message": "trying out Elasticsearch", "context": "foo"}
+  - do:
+      bulk:
+        index: logs-my_app-default
+        refresh: true
+        body: |
+          {"create":{"_id": "0"}}
+          {"message": "trying out Elasticsearch", "context": "foo"}
+  - do:
+      snapshot.create:
+        repository: my_repository
+        snapshot: my_snapshot_2099.05.06
+        wait_for_completion: true
+'''
+
 // Fake sec logs data used by EQL search
   setups['atomic_red_regsvr32'] = setups['my_data_stream'] + '''
   - do:

docs/reference/high-availability/backup-and-restore-security-config.asciidoc

@@ -162,115 +162,3 @@ of the users in the native realm and revoke API keys.
 * If there are indications that the snapshot has been tampered with, do not
 restore it. There is currently no option for the restore process to detect
 malicious tampering.
-
-[[restore-security-configuration]]
-=== Restore a cluster's security configuration
-++++
-<titleabbrev>Restore the security configuration</titleabbrev>
-++++
-
-NOTE: You can restore a snapshot of the `.security` index only if it was
-created in a previous minor version in the same major version. The last minor
-version of every major release can convert and read formats of the index for
-both its major version and the next one.
-
-When you restore security configuration you have the option of doing a complete
-restore of *all* configurations, including non-security ones, or to only restore
-the contents of the `.security` index. As described in
-<<backup-security-index-configuration>>, the second option comprises only
-resource-type configurations. The first option has the advantage of restoring
-a cluster to a clearly defined state from a past point in time. The second option
-touches only security configuration resources, but it does not completely restore
-the {security-features}.
-
-To restore your security configuration from a backup, first make sure that the
-repository holding `.security` snapshots is installed:
-
-[source,console]
---------------------------------------------------
-GET /_snapshot/my_backup
---------------------------------------------------
-// TEST[continued]
-
-[source,console]
---------------------------------------------------
-GET /_snapshot/my_backup/snapshot_1
---------------------------------------------------
-// TEST[continued]
-
-Then log in to one of the node hosts, navigate to {es} installation directory,
-and follow these steps:
-
-. Add a new user with the `superuser` built-in role to the
-<<file-realm,file realm>>.
-+
---
-For example, create a user named `restore_user`:
-[source,shell]
---------------------------------------------------
-bin/elasticsearch-users useradd restore_user -p password -r superuser
---------------------------------------------------
---
-
-. Using the previously created user, delete the existing `.security-6` or
-`.security-7` index.
-+
---
-[source,shell]
---------------------------------------------------
-curl -u restore_user -X DELETE "localhost:9200/.security-*"
---------------------------------------------------
-// NOTCONSOLE
-
-WARNING: After this step any authentication that relies on the `.security`
-index will not work. This means that all API calls that authenticate with
-native or reserved users will fail, as will any user that relies on a native role.
-The file realm user we created in the step above will continue to work
-because it is not stored in the `.security` index and uses the built-in
-`superuser` role.
-
---
-
-. Using the same user, restore the `.security` index from the snapshot.
-+
---
-[source,shell]
---------------------------------------------------
- curl -u restore_user -X POST "localhost:9200/_snapshot/my_backup/snapshot_1/_restore" -H 'Content-Type: application/json' -d'
- {
-    "indices": ".security-*",
-    "include_global_state": true <1>
- }
- '
---------------------------------------------------
-// NOTCONSOLE
-
-<1> The `include_global_state: true` is mandatory only for a complete restore.
-This will restore the global cluster metadata, which contains configuration
-information for the complete cluster. If you set this to `false`, it recovers
-only the contents of the `.security` index, such as usernames and password
-hashes, API keys, application privileges, role and role mapping definitions.
---
-
-. Optionally, if you need to review and override the settings that were included
-in the snapshot (by the `include_global_state` flag), cherry-pick and
-<<cluster-update-settings,apply the persistent settings>> that you
-<<backup-cluster-configuration, have extracted>> with the
-`GET _cluster/settings` API.
-
-. If you pursue a complete point in time restore of the cluster, you also have
-to restore configuration files. Again, this will restore non-security settings as
-well.
-+
---
-This entails a straight-up filesystem copy of the backed up configuration files,
-overwriting the contents of `$ES_PATH_CONF`, and restarting the node. This
-needs to be done on *every node*. Depending on the extent of the differences
-between your current cluster configuration and the restored configuration, you
-may not be able to perform a rolling restart. If you are performing a full
-restore of your configuration directory, we recommend a full cluster restart as
-the safest option. Alternatively, you may wish to restore your configuration
-files to a separate location on disk and use file comparison tools to review
-the differences between your existing configuration and the restored
-configuration.
---

docs/reference/high-availability/backup-cluster.asciidoc

@@ -9,14 +9,10 @@ To have a complete backup for your cluster:
 . <<backup-cluster-configuration,Back up the cluster configuration>>
 . <<security-backup,Back up the security configuration>>
 
-To restore your cluster from a backup:
-
-. <<restore-cluster-data,Restore the data>>
-. <<restore-security-configuration,Restore the security configuration>>
+To restore your cluster from a backup, see <<restore-entire-cluster>>.
 
 
 
 include::backup-cluster-data.asciidoc[]
 include::backup-cluster-config.asciidoc[]
 include::backup-and-restore-security-config.asciidoc[]
-include::restore-cluster-data.asciidoc[]

docs/reference/high-availability/restore-cluster-data.asciidoc

@@ -3,6 +3,10 @@
 
 The following pages have moved or been deleted.
 
+[role="exclude",id="restore-cluster-data"]
+=== Restore a cluster's data
+
+See <<restore-entire-cluster>>.
 [role="exclude",id="alias"]
 === Aliases
 
@@ -767,7 +771,7 @@ See <<snapshots-repository-plugins>>.
 [role="exclude",id="_changing_index_settings_during_restore"]
 ==== Change index settings during restore
 
-See <<change-index-settings-during-restore>>.
+See <<restore-snapshot-api-index-settings>>.
 
 [role="exclude",id="restore-snapshot"]
 === Restore snapshot

docs/reference/redirects.asciidoc

@@ -64,29 +64,26 @@ POST /_snapshot/my_repository/my_snapshot/_restore
 [[restore-snapshot-api-prereqs]]
 ==== {api-prereq-title}
 
-* If the {es} {security-features} are enabled, you must have the `manage`
-<<privileges-list-cluster,cluster privilege>> to use this API.
+// tag::restore-prereqs[]
+* If you use the {es} security features, you must have the `manage` or
+`cluster:admin/snapshot/*` cluster privilege to restore a snapshot.
 
-[[restore-snapshot-api-desc]]
-==== {api-description-title}
+* You can only restore a snapshot to a running cluster with an elected
+<<master-node,master node>>. The snapshot's repository must be
+<<snapshots-register-repository,registered>> and available to the cluster.
 
-Use the restore snapshot API to restore a snapshot of a cluster, including all data streams and indices in the snapshot. If you do not want to restore the entire snapshot, you can select specific data streams or indices to restore.
+* The snapshot and cluster versions must be compatible. See
+<<snapshot-restore-version-compatibility>>.
 
-You can run the restore operation on a cluster that contains an elected
-<<master-node,master node>> and has data nodes with enough capacity to accommodate the snapshot
-you are restoring. Existing indices can only be restored if they are
-<<indices-close,closed>> and have the same number of shards as the indices in
-the snapshot. The restore operation automatically opens restored indices if
-they were closed and creates new indices if they do not exist in the cluster.
+* If you restore a data stream, ensure the cluster contains a
+<<create-index-template,matching index template>> with data stream enabled.
+Without a matching index template, a data stream can't roll over or create
+backing indices.
 
-If a data stream is restored, its aliases and backing indices are also restored.
-Alternatively, you can restore individual backing indices without restoring an
-entire data stream. If you restore individual backing indices, they are not
-automatically added to any existing data stream. For example, if only the
-`.ds-logs-2099.03.08-00003` backing index is restored from a snapshot, it is not
-automatically added to the existing `logs` data stream.
-
-include::{es-ref-dir}/snapshot-restore/restore-snapshot.asciidoc[tag=index-settings-data-stream-warning]
+* If your snapshot contains data from App Search or Workplace Search, ensure
+you've restored the {enterprise-search-ref}/encryption-keys.html[Enterprise
+Search encryption key] before restoring the snapshot.
+// end::restore-prereqs[]
 
 [[restore-snapshot-api-path-params]]
 ==== {api-path-parms-title}
@@ -107,7 +104,7 @@ include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=master-timeout]
 `wait_for_completion`::
 (Optional, Boolean) If `true`, the request returns a response when the restore
 operation completes. The operation is complete when it finishes all attempts to
-<<_monitoring_restore_operations,recover primary shards>> for restored indices.
+<<monitor-restore,recover primary shards>> for restored indices.
 This applies even if one or more of the recovery attempts fail.
 +
 If `false`, the request returns a response when the restore
@@ -118,14 +115,15 @@ operation initializes. Defaults to `false`.
 ==== {api-request-body-title}
 
 `ignore_unavailable`::
-(Optional, Boolean)
-If `false`, the request returns an error for any data stream or index that is missing or closed. Defaults to `false`.
-+
-If `true`, the request ignores data streams and indices in `indices` that are missing or closed.
+(Optional, Boolean) If `true`, the request ignores any index or data stream in
+`indices` that's missing from the snapshot. If `false`, the request returns an
+error for any missing index or data stream. Defaults to `false`.
 
 `ignore_index_settings`::
-(Optional, string)
-A comma-separated list of index settings that should not be restored from a snapshot.
+(Optional, string or array of strings) Index settings to not restore from the snapshot. You can't use this option to ignore <<index-number-of-shards,`index.number_of_shards`>>.
++
+For data streams, this option only applies to restored backing indices. New
+backing indices are configured using the data stream's matching index template.
 
 `include_aliases`::
 (Optional, Boolean)
@@ -171,23 +169,15 @@ By default, all available feature states will be restored if `include_global_sta
 
 [[restore-snapshot-api-index-settings]]
 `index_settings`::
-(Optional, string)
-A comma-separated list of settings to add or change in all restored indices. Use this parameter to override index settings when restoring snapshots.
+(Optional, object) Index settings to add or change in restored indices,
+including backing indices. You can't use this option to change
+<<index-number-of-shards,`index.number_of_shards`>>.
 +
-For data streams, these index settings are applied to the restored backing
-indices.
-+
-For more information regarding all the different index-level settings
-that you can specify, see
-<<index-modules,index modules>>.
+For data streams, this option only applies to restored backing indices. New
+backing indices are configured using the data stream's matching index template.
 
 `indices`::
-(Optional, string)
-A comma-separated list of data streams and indices to restore from the snapshot.
-<<multi-index,Multi-index syntax>> is supported.
-+
-By default, a restore operation includes all data streams and indices in the snapshot. If this
-argument is provided, the restore operation only includes the specified data streams and indices.
+(Optional, string or array of strings) Comma-separated list of indices and data streams to restore. Supports <<api-multi-index,multi-target syntax>>. Defaults to all indices and data streams in the snapshot, including system indices.
 
 [[restore-snapshot-api-partial]]
 `partial`::
@@ -205,10 +195,6 @@ Defines a rename pattern to apply to restored data streams and indices. Data str
 +
 The rename pattern is applied as defined by the regular expression that
 supports referencing the original text, according to the https://docs.oracle.com/javase/8/docs/api/java/util/regex/Matcher.html#appendReplacement-java.lang.StringBuffer-java.lang.String-[`appendReplacement`] logic.
-+
-The request will fail if two or more data streams or indices will be renamed into the same name.
-+
-include::{es-ref-dir}/snapshot-restore/restore-snapshot.asciidoc[tag=rename-restored-data-stream-tag]
 
 [[restore-snapshot-api-rename-replacement]]
 `rename_replacement`::

docs/reference/snapshot-restore/apis/restore-snapshot-api.asciidoc

@@ -118,8 +118,8 @@ understand the time requirements before proceeding.
 
 include::register-repository.asciidoc[]
 include::take-snapshot.asciidoc[]
-include::restore-snapshot.asciidoc[]
 include::monitor-snapshot-restore.asciidoc[]
 include::delete-snapshot.asciidoc[]
+include::restore-snapshot.asciidoc[]
 include::../slm/index.asciidoc[]
 include::../searchable-snapshots/index.asciidoc[]

docs/reference/snapshot-restore/index.asciidoc

@@ -1,8 +1,5 @@
 [[snapshots-monitor-snapshot-restore]]
-== Monitor snapshot and restore progress
-++++
-<titleabbrev>Monitor snapshot and restore</titleabbrev>
-++++
+== Monitor snapshot progress
 
 Use the <<get-snapshot-api,get snapshot API>> or the
 <<get-snapshot-status-api,get snapshot status API>> to monitor the
@@ -132,25 +129,9 @@ if not currently running:
 GET /_snapshot/my_backup/snapshot_1/_status
 -----------------------------------
 
-[discrete]
-=== Monitoring restore operations
-
-The restore process piggybacks on the standard recovery mechanism of
-{es}. As a result, standard recovery monitoring services can be used
-to monitor the state of restore. When the restore operation starts, the
-cluster typically goes into `yellow` state because the restore operation works
-by recovering primary shards of the restored indices. After the recovery of the
-primary shards is completed, {es} switches to the standard replication
-process that creates the required number of replicas. When all required
-replicas are created, the cluster switches to the `green` states.
-
-The cluster health operation provides only a high level status of the restore process. It's possible to get more
-detailed insight into the current state of the recovery process by using <<indices-recovery, index recovery>> and
-<<cat-recovery, cat recovery>> APIs.
-
 [discrete]
 [[get-snapshot-stop-snapshot]]
-=== Stop snapshot and restore operations
+=== Stop snapshot operations
 To stop a currently running snapshot that was started by mistake or is taking unusually long, use
 the <<delete-snapshot-api,delete snapshot API>>.
 This operation checks whether the deleted snapshot is currently running. If it is, the delete snapshot operation stops
@@ -161,10 +142,6 @@ that snapshot before deleting the snapshot data from the repository.
 DELETE /_snapshot/my_backup/snapshot_1
 -----------------------------------
 
-The restore operation uses the standard shard recovery mechanism. Therefore, any currently running restore operation can
-be canceled by deleting data streams and indices that are being restored. Data for all deleted data streams and indices will be removed
-from the cluster as a result of this operation.
-
 [discrete]
 [[get-snapshot-cluster-blocks]]
 === Effect of cluster blocks on snapshot and restore