Fixed rounding of bounds in scaled float comparison

[original-brownbear]

This fixes #27189 I think:

• Fixed by switching the order of operations between nextDown, nextUp and multiplication by the scaling factor to reduce rounding errors. Now the error incurred by nextUp and nextDown doesn't propagate into the multiplication by the scaling factor, whereas without this change the error incurred when multiplying two double can cancel out the Math.nextDown step as seen in the example below
  • Added unit tests for the example in scaled_float has precision problem #27189 in which we were hit by

Math.floor(Math.nextDown(0.1) * 100.0) = 10.0

while

Math.floor(Math.nextDown(0.1 * 100.0)) = 9.0

[jpountz]

I left a comment about testing more cases, but your change looks good to me.

[jpountz]

I expanded your assertions a bit if you are ok with them:

    public void testRoundsUpperBoundCorrectly() {
        ScaledFloatFieldMapper.ScaledFloatFieldType ft = new ScaledFloatFieldMapper.ScaledFloatFieldType();
        ft.setName("scaled_float");
        ft.setScalingFactor(100.0);
        Query scaledFloatQ = ft.rangeQuery(null, 0.1, true, false, null);
        assertEquals("scaled_float:[-9223372036854775808 TO 9]", scaledFloatQ.toString());
        scaledFloatQ = ft.rangeQuery(null, 0.1, true, true, null);
        assertEquals("scaled_float:[-9223372036854775808 TO 10]", scaledFloatQ.toString());
        scaledFloatQ = ft.rangeQuery(null, 0.095, true, false, null);
        assertEquals("scaled_float:[-9223372036854775808 TO 9]", scaledFloatQ.toString());
        scaledFloatQ = ft.rangeQuery(null, 0.095, true, true, null);
        assertEquals("scaled_float:[-9223372036854775808 TO 9]", scaledFloatQ.toString());
        scaledFloatQ = ft.rangeQuery(null, 0.105, true, false, null);
        assertEquals("scaled_float:[-9223372036854775808 TO 10]", scaledFloatQ.toString());
        scaledFloatQ = ft.rangeQuery(null, 0.105, true, true, null);
        assertEquals("scaled_float:[-9223372036854775808 TO 10]", scaledFloatQ.toString());
    }

    public void testRoundsLowerBoundCorrectly() {
        ScaledFloatFieldMapper.ScaledFloatFieldType ft = new ScaledFloatFieldMapper.ScaledFloatFieldType();
        ft.setName("scaled_float");
        ft.setScalingFactor(100.0);
        Query scaledFloatQ = ft.rangeQuery(-0.1, null, false, true, null);
        assertEquals("scaled_float:[-9 TO 9223372036854775807]", scaledFloatQ.toString());
        scaledFloatQ = ft.rangeQuery(-0.1, null, true, true, null);
        assertEquals("scaled_float:[-10 TO 9223372036854775807]", scaledFloatQ.toString());
        scaledFloatQ = ft.rangeQuery(-0.095, null, false, true, null);
        assertEquals("scaled_float:[-9 TO 9223372036854775807]", scaledFloatQ.toString());
        scaledFloatQ = ft.rangeQuery(-0.095, null, true, true, null);
        assertEquals("scaled_float:[-9 TO 9223372036854775807]", scaledFloatQ.toString());
        scaledFloatQ = ft.rangeQuery(-0.105, null, false, true, null);
        assertEquals("scaled_float:[-10 TO 9223372036854775807]", scaledFloatQ.toString());
        scaledFloatQ = ft.rangeQuery(-0.105, null, true, true, null);
        assertEquals("scaled_float:[-10 TO 9223372036854775807]", scaledFloatQ.toString());
    }

[original-brownbear]

@jpountz thanks for reviewing, I like adding some more assertions :) I'll just add those to this PR?

[jpountz]

Yes please!

[original-brownbear]

@jpountz added your assertions :)

[dakrone]

Thanks @original-brownbear, I'll wait for CI to finish and then merge this

[original-brownbear]

Rebased this one just now since it failed the BWC tests here https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+pull-request/5329/console

[original-brownbear]

@dakrone now it's green :)