Coerce decimal strings for whole number types by truncating the decimal part #25835
Conversation
|
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
There was a problem hiding this comment.
This does not preserve the precision of large numbers that have a decimal part and exceed the fractional bits of double but are within min/max Long. For example, while both 4115420654264075766 and "4115420654264075766" will be indexed as 4115420654264075766, something like "4115420654264075766.1" will not be indexed as 4115420654264075766 due to String -> Double -> Long conversion.
Do we want to try to do something a bit more clever to handle this edge case?
There was a problem hiding this comment.
I think I'd just put a comment that we might not fail in all cases, but I don't think I would try to address it?
There was a problem hiding this comment.
Do you mean a comment in the code or in the documentation? (i.e. a warning here https://www.elastic.co/guide/en/elasticsearch/reference/current/coerce.html)
There was a problem hiding this comment.
I mean in the code but just noticed there was one already
There was a problem hiding this comment.
I removed this test since this change breaks it but I don't think that this test makes sense in the current state. For example, before this change that test would also break if you changed it from "2.0" to a decimal literal like 2.5. Now it is consistent (it will accept and truncate decimals, whether you quote them or not).
There was a problem hiding this comment.
Sorry I deleted your comment by mistake, so I am adding it back:
This only rejects coerce=false values for numbers with decimals but it won't reject strings, which seems to run contrary to how coercion is supposed to work. I did not change this behaviour. I just wanted to call it out as inconsistent with how the parse methods in AbstractXContentParser work, should this be changed?
There was a problem hiding this comment.
Agreed it should be changed to be consistent, but let's do it in a separate PR? Thinking more about it, I'm wondering whether we should remove the coerce option instead. I opened #25861.
|
@elasticmachine Please test it. |
jpountz
left a comment
jpountz
left a comment
There was a problem hiding this comment.
It looks good to me, thanks for working on it.
There was a problem hiding this comment.
Agreed it should be changed to be consistent, but let's do it in a separate PR? Thinking more about it, I'm wondering whether we should remove the coerce option instead. I opened #25861.
There was a problem hiding this comment.
I think I'd just put a comment that we might not fail in all cases, but I don't think I would try to address it?
scottsom
force-pushed
the
decimal_string_coerce
branch
from
92d11dc to
a82bd08
Compare
|
Added |
|
@elasticmachine Please test it. |
…al part This changes makes it so you can index a value like "1.0" or "1.1" into whole number field types like byte and integer. Without this change then the above values would have resulted in an error, even with coerce set to true. Closes elastic#25819
scottsom
force-pushed
the
decimal_string_coerce
branch
from
a82bd08 to
27371b0
Compare
|
The build failure seems transient and/or unrelated to this change. I've rebased onto the latest master since I see some newer changes that appear to be improving test stability. |
|
@elasticmachine Test it please. |
There was a problem hiding this comment.
I mean in the code but just noticed there was one already
|
Thank you @scottsom ! |
jpountz
added
:Search Foundations/Mapping
5 participants
This changes makes it so you can index a value like "1.0" or "1.1" into whole
number field types like byte and integer. Without this change then the above
values would have resulted in an error, even with coerce set to true.
Closes #25819