Remove content detection mechanism from REST endpoints

[jasontedor]

Today, some Elasticsearch REST endpoints (e.g., update index settings API) attempt to determine the content type (e.g., JSON vs. YAML) of the request body with a rudimentary detection mechanism. This mechanism breaks down on YAML bodies that contain braces because the content is inappropriately detected as JSON. Instead, this mechanism should not be used, Elasticsearch should default to JSON for all request bodies, and possibly obey the Content-Type header for allowing YAML.

Relates #19366

[javanna]

I was under the impression that we already obey to the Content-Type header, and it's a bug if some apis don't. Seems like we do things differently only for apis that handle settings. I noticed that we support "properties" format for settings, which may be the reason behind this weird choice... wondering who uses/remembers that we support properties format there.

[jasontedor]

I was under the impression that we already obey to the Content-Type header, and it's a bug if some apis don't.

Sadly we do not always. 😢

I noticed that we support "properties" format for settings, which may be the reason behind this weird choice... wondering who uses/remembers that we support properties format there.

I think that should be removed as part of closing this issue. 😄

[javanna]

Sadly we do not.

It seems like we do apart from where we accept settings in the body? Or are there other REST endpoints where we try and detect content type?

I think that should be removed as part of closing this issue.

Sure, why not. Maybe it should be brought up before it gets removed though :)

[clintongormley]

I noticed that we support "properties" format for settings, which may be the reason behind this weird choice... wondering who uses/remembers that we support properties format there.

Not documented. Please rip it out!

[javanna]

Not documented. Please rip it out!

Perfect that makes it a no-brainer, with properties out of the way it makes no sense to do any auto-detection I think. We should do for settings what we already do for any other request I believe.

[jasontedor]

We can completely remove support for properties. I opened #19391.

[anti-social]

If you removed --- requirement YAML parser could parse JSON since YAML is a superset of JSON.

[clintongormley]

To quote from https://metacpan.org/pod/JSON::XS#JSON-and-YAML

YAML has hardcoded limits on (simple) object key lengths that JSON doesn't have and also has different and incompatible unicode character escape syntax, so you should make sure that your hash keys are noticeably shorter than the 1024 "stream characters" YAML allows and that you do not have characters with codepoint values outside the Unicode BMP (basic multilingual page). YAML also does not allow / sequences in strings

[popravich]

ok, but what about checking Content-Type header?
YAML format suits our flow the best -- its easier to read, write, add comments and maintain at all.
It would be really great if you guys atleast keep current YAML support.

[jasontedor]

@popravich That's the plan.

[javanna]

I care to highlight that we already take into account the Content-Type header. I think the title of this issue is misleading as it makes people think that we guess the content type all the time, while we (used to) have this detection in place only for apis that accept settings in their request body. The general approach is to take Content-Type into account and that's always been the case apart from the mentioned exceptions.