Snapshots does not include closed indices without explicitly setting expand_wildcards to "all"

[tomhe]

Elasticsearch version (bin/elasticsearch --version):

Version: 7.10.1, Build: default/docker/1c34507e66d7db1211f66f3513706fdf548736aa/2020-12-05T01:00:33.671820Z, JVM: 15.0.1

Plugins installed: []

JVM version (java -version):

OS version (uname -a if on a Unix-like system):

Linux a3406564826d 5.3.0-46-generic #38~18.04.1-Ubuntu SMP Tue Mar 31 04:17:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:

The create snapshot documentation states that

The expand_wildcards option can be used to control whether hidden and closed
indices will be included in the snapshot, and defaults to all.

However, the default setting is not respected when a snapshot is created.

Creating a snapshot and relying on default values does not include closed indices:

POST /_snapshot/repo/snapshot

Explicitly setting expand_wildcards to all includes closed indices:

POST /_snapshot/repo/snapshot
{
  "expand_wildcards": "all"
}

There is no way to control expand_wildcards when using a snapshot policy to
automatically create snapshots.

I expect closed indices to be included in the snapshot without explicitly
providing the default setting.

Steps to reproduce:

Please include a minimal but complete recreation of the problem,
including (e.g.) index creation, mappings, settings, query etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

1. Create a docker-compose.yml file:

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.1
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - path.repo=/tmp
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - elastic

  kib01:
    image: docker.elastic.co/kibana/kibana:7.10.1
    container_name: kib01
    ports:
      - 5601:5601
    environment:
      ELASTICSEARCH_URL: http://es01:9200
      ELASTICSEARCH_HOSTS: http://es01:9200
    networks:
      - elastic

volumes:
  data01:
    driver: local

networks:
  elastic:
    driver: bridge

2. Bring up the cluster:

docker-compose up

3. Bring up the dev console in Kibana running on 5601.
4. Create snapshot repository:

PUT /_snapshot/repo
{
 "type": "fs",
 "settings": {
   "location": "/tmp"
 }
}

=>

{
  "acknowledged" : true
}

5. Create an index abc123:

PUT abc123

=>

{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "abc123"
}

6. Close the index abc123:

POST abc123/_close

=>

{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "indices" : {
    "abc123" : {
      "closed" : true
    }
  }
}

7. Create snapshot using default settings:

POST /_snapshot/repo/snapshot_1?wait_for_completion=true

=>

{
  "snapshot" : {
    "snapshot" : "snapshot_1",
    "uuid" : "YKOsNh0DTFya9t_eIIfgSw",
    "version_id" : 7100199,
    "version" : "7.10.1",
    "indices" : [
      ".apm-agent-configuration",
      "ilm-history-3-000001",
      ".kibana-event-log-7.10.1-000001",
      ".kibana_1",
      ".kibana_task_manager_1",
      ".apm-custom-link"
    ],
    "data_streams" : [ ],
    "include_global_state" : true,
    "state" : "SUCCESS",
    "start_time" : "2020-12-18T10:46:38.892Z",
    "start_time_in_millis" : 1608288398892,
    "end_time" : "2020-12-18T10:46:39.092Z",
    "end_time_in_millis" : 1608288399092,
    "duration_in_millis" : 200,
    "failures" : [ ],
    "shards" : {
      "total" : 6,
      "failed" : 0,
      "successful" : 6
    }
  }
}

8. Notice that the index abc123 is missing in the response.
9. Create a snapshot and exlicitly set expand_wildcards to all:

POST /_snapshot/repo/snapshot_2?wait_for_completion=true
{
  "expand_wildcards": "all"
}

=>

{
  "snapshot" : {
    "snapshot" : "snapshot_2",
    "uuid" : "-VtCDmsVSOiiVcYpZAHs_g",
    "version_id" : 7100199,
    "version" : "7.10.1",
    "indices" : [
      ".apm-agent-configuration",
      "ilm-history-3-000001",
      ".kibana-event-log-7.10.1-000001",
      ".kibana_1",
      "abc123",
      ".kibana_task_manager_1",
      ".apm-custom-link"
    ],
    "data_streams" : [ ],
    "include_global_state" : true,
    "state" : "SUCCESS",
    "start_time" : "2020-12-18T10:49:12.418Z",
    "start_time_in_millis" : 1608288552418,
    "end_time" : "2020-12-18T10:49:12.619Z",
    "end_time_in_millis" : 1608288552619,
    "duration_in_millis" : 201,
    "failures" : [ ],
    "shards" : {
      "total" : 7,
      "failed" : 0,
      "successful" : 7
    }
  }
}

10. Notice that the index abc123 is included in the response.

[elasticmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[tlrx]

Starting the next major version (v8.0) closed indices will be included by default in snapshots (see #39644). As you noticed in 7.x the expand_wildcards value must explicitly set to all.

We should correct the documentation for 7.x.

[stefnestor]

🙏🏼 @jrodewig / ES Docs team, this is stating this line from this doc page needs updated:

[elasticmachine]

Pinging @elastic/es-docs (Team:Docs)

[jrodewig]

Closed by #75432