es.xcontent.strict_duplicate_detection warning required

[damianpfister]

Elasticsearch version (bin/elasticsearch --version): 7.7.0

Plugins installed: [N/A]

JVM version (java -version): OpenJDK 64-Bit Server VM 14u36

OS version (uname -a if on a Unix-like system): RHEL 7

Description of the problem including expected versus actual behavior:
When a cluster has indices with duplicate fields prior to 6.x and es.xcontent.strict_duplicate_detection is set to false in order to by-pass the strict duplicate checks, there is no warning this is depricated. Then when upgraded to 7.x (with the same setting) this setting doesn't work - we removed that setting in 7.0 (#34588) and it was listed under the "Enhancements" of 7.0.0-alpha1 (https://www.elastic.co/guide/en/elasticsearch/reference/current/release-notes-7.0.0-alpha1.html#enhancement-7.0.0-alpha1).

Version 7.x should not allow the cluster to start with this setting configured and perhaps the upgrade assistant should detect the duplicates (am assuming it didn't in this case).

Steps to reproduce:
Index in 5.x with duplicate fields, upgrade to 6.x and disable es.xcontent.strict_duplicate_detection (works) then upgrade to 7.x and indexing no longer works

Provide logs (if relevant): N/A

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Core)

[cjcenizal]

@damianpfister Can I echo this back to make sure I understand correctly?

1. Elasticsearch fundamentally disallows duplicate fields.
2. The strict_duplicate_detection setting allowed people to alter this behavior to allow duplicate fields in 6.x and prior.
3. This setting is removed in 7.0, causing people who had been depending upon it to be surprised by duplicate field errors when they upgrade to 7.x.

The proposed solution is to:

1. Update Elasticsearch 7.x to emit an error and disallow the cluster from starting if this setting is configured.
2. Update the Kibana 7.x Upgrade Assistant to support the user in removing duplicates from their indices.

Is all of this accurate? If this is the case, then I will create a Kibana issue to track 2 above. We should ensure both changes in ES and Kibana are released in the same versions. Otherwise we risk driving users into a state in which their cluster contains indices with duplicate fields and they have no guidance for identifying or addressing the problem.

I have a number of questions around this:

• What is the typical process for identifying and resolving duplicate fields in a user's indices?
• How does a user create duplicate fields in an index?
• How are these errors typically surfaced?

[williamrandolph]

I am preparing a pull request that will log a warning for this case. When the core-infra team discussed this issue, we wanted to start with a deprecation warning.

One thing to note is that es.xcontent.strict_duplicate_detection and es.json.strict_duplicate_detection were never documented Elasticsearch settings which could be retrieved from the settings endpoint. They were Java arguments that were accessed as Java system properties. So in order to detect this property's presence, you have to do something like this:

curl -s --user elastic:password localhost:9200/_nodes/ | jq '.nodes[].jvm.input_arguments'

[williamrandolph]

Beginning in 7.12, there will be a deprecation log reading: The Java option es.xcontent.strict_duplicate_detection is set to [true]; this option is deprecated and non-functional and should be removed from Java configuration.

We didn't believe that this issue was severe enough to block node startup, but please feel free to re-open the ticket if blocking node startup is important.