Synthetic _source: support ignore_malformed

[felixbarny]

As part of LX, we want to make log indices more lenient. Therefore, we're planning on using ignore_malformed. See also #88777. While using synthetic source is not a requirement for LX, we should think about how we could eventually leverage synthetic source to save space.

One of the blockers is that synthetic source doesn't have support for ignore_malformed. That's because malformed values will only be available in the _source. But if the source is synthetic, malformed values would just be lost.

We could to something similar to #89466 and store malformed values in a hidden stored field. One of the challenges is that stored fields are strictly typed (IINM) but the malformed value could be of any type. But we could just always store the malformed value in a string field, in the exact form that was provided in the JSON document. For example "42" (note the quotes), true, 42. When re-constructing the _source, we can just use that value and append it to the JSON document. Just an idea :)

cc @nik9000

[elasticsearchmachine]

Pinging @elastic/es-analytics-geo (Team:Analytics)

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[nik9000]

Began in #90038

[javanna]

heya @nik9000 do you remember what is left to do here?

[felixbarny]

AFAICS, Nik added support for the ip field type but others are missing. I guess the first step would be to create a list of all field types and tick the one(s) off that are supported already and then work on implementing support for the rest.

We have recently made the change to use ignore_malformed by default for logs: #95329.

[felixbarny]

create a list of all field types and tick the one(s) off that are supported already and then work on implementing support for the rest.

Nvm. this already exists here: #86603

[nik9000]

heya @nik9000 do you remember what is left to do here?

Wow I missed this comment. Sorry.

I left off trying to build ignore_malformed for geo_point which is especially complex because it's a json object and not a single field. So we have to "go back" if we hit a malformed object. I think I had it working, but the implementation made me sad: #90777 . I picked geo_point at the time because it was the simplest example of the "complex" fields. But I think the way forward is clear for most of the other field types. Just not the json object ones.