Removing Remote Cluster from Cluster Settings requires array format

[pickypg]

When trying to remove a cluster from my cross-cluster search list, I kept failing to remove the cluster by using the documented approach:

PUT /_cluster/settings
{
  "persistent": {
    "search.remote.local_cluster.seeds": null
  }
}

even though it claims to successfully use the settings:

{
  "acknowledged": true,
  "persistent": {},
  "transient": {}
}

However, to actually remove the seeds (and therefore the remote cluster), you need to wrap the null in [] brackets.

PUT /_cluster/settings
{
  "persistent": {
    "search": {
      "remote": {
        "my_cluster": {
          "seeds": ["127.0.0.1:9300"] 
        }
      }
    }
  }
}

PUT /_cluster/settings
{
  "persistent": {
    "search.remote.my_cluster.seeds": [ null ]
  }
}

For what it's worth, simply passing [] does not work.

[jasontedor]

Sadly this is a case of array settings biting us again because they get converted to keys key.0, key.1, etc.

[javanna]

there is an open community PR for it, @jasontedor would you mind having a look at it?

[jasontedor]

@javanna It looks like @rjernst has taken it up and we will need a different approach (namely addressing the underlying issue with how we handle multi-valued settings).

[inqueue]

Another report of this issue was received today. Wrapping the null in brackets does not work if there are multiple seeds. A subsequent request is required to make it go away:

GET /_cluster/settings
{
  "persistent": {
    "search": {
      "remote": {
        "cluster_one": {
          "seeds": [
            "127.0.0.1:9301",
            "127.0.0.1:9302"
          ]
        }
      }
    }
  },
  "transient": {}
}

PUT /_cluster/settings
{
  "persistent": {
    "search.remote.cluster_one.seeds": [null]
  }
}

GET /_cluster/settings
{
  "persistent" : {
    "search" : {
      "remote" : {
        "cluster_one" : {
          "seeds" : {
            "1" : "127.0.0.1:9302"
          }
        }
      }
    }
  },
  "transient" : { }
}

PUT /_cluster/settings
{
  "persistent": {
    "search.remote.cluster_one.seeds.1": null
  }
}

GET /_cluster/settings
{
  "persistent" : { },
  "transient" : { }
}

[jakommo]

@javanna you mentioned this was solved in master already, but it's still open. Is it this PR #26878 ?

[javanna]

I just did some testing, this is fixed in 6.x and master, still happens in 6.0 and 5.6. I do think that #26878 fixed it, but I am not 100% sure. Closing.