Reject out of range numeric values at index time

[colings86]

Currently if you index a document that contains a numeric value which is out of the range of the numeric type for the field we accept the document and index an infinite value. This can lead to issues like #23003.

Instead we should reject value which are out of range for the selected numeric type (we do this for scaled_float but not currently for float, double and half_float, and we should check integer and long types). For completeness we should also reject explicit infinite values and NaN values in case a script produces these values. JSON does not have a concept of NaN or infinite values so they could only be explicitly inserted by scripts and never from standard index requests.

An open question is whether ignore_malformed should ignore out of range values?

[jpountz]

+1

An open question is whether ignore_malformed should ignore out of range values?

To me this option means "index it if you can, but please never fail, I'd rather have my data not searchable" so I believe we should simply ignore out-of-range values when this option is set to true. Separately maybe we should reopen the discussion about its removal since it makes it too easy to have data not searchable silently, which is barely better than a data loss.

[fred84]

@colings86 May I try to solve this issue?

[colings86]

@fred84 that would be great if you are able to work on this. Please do let me know if you have any questions, and feel free to add me as a reviewer when you have a PR ready

[fred84]

@colings86 I think it is suitable to duplicate logic from subclasses of Lucene's Field for handling out-of-bound values in half float, float and double. Example: fred84/elasticsearch@2d0e5c1 . Am I going in right direction?

[jpountz]

I think we should check that Double.isFinite is true for doubles, Float.isFinite is true for floats and that Float.isFinite(HalfFloatPoint.shortBitsToHalfFloat(HalfFloatPoint.halfFloatToShortBits(f))) is true for half floats (converting to a short and back to a float simulates the rounding that happens at index time).

[cbuescher]

I think this was adressed by #25826 and can be closed.
@colings86 please reopen if you think there is anything left to do here.