Skip to content

Conversation

@yjshen
Copy link
Member

@yjshen yjshen commented Aug 13, 2015

I made a mistake in #8049 by casting literal value to attribute's data type, which would cause simply truncate the literal value and push a wrong filter down.

JIRA: https://issues.apache.org/jira/browse/SPARK-9927

@liancheng
Copy link
Contributor

@yjshen Could you provide a snippet that reproduces the error you mentioned?

@yjshen
Copy link
Member Author

yjshen commented Aug 13, 2015

select * from inttypes where A > 2147483648, it would produce GreaterThan(Cast(A, LongType), Literal(2147483648L, LongType), in #8049 I just cast the long value to IntType, therefore the pushed down filter became A > -2147483648.

@SparkQA
Copy link

SparkQA commented Aug 13, 2015

Test build #40723 has finished for PR 8157 at commit db15678.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class QRDecomposition[QType, RType](Q: QType, R: RType)

@liancheng
Copy link
Contributor

@yjshen Thanks, I'm merging this to master and branch-1.5 and reopening SPARK-9182.

asfgit pushed a commit that referenced this pull request Aug 13, 2015
I made a mistake in #8049 by casting literal value to attribute's data type, which would cause simply truncate the literal value and push a wrong filter down.

JIRA: https://issues.apache.org/jira/browse/SPARK-9927

Author: Yijie Shen <[email protected]>

Closes #8157 from yjshen/rever8049.

(cherry picked from commit d0b1891)
Signed-off-by: Cheng Lian <[email protected]>
@asfgit asfgit closed this in d0b1891 Aug 13, 2015
@yjshen
Copy link
Member Author

yjshen commented Aug 13, 2015

@liancheng , I'm really sorry for my mistake.

@liancheng
Copy link
Contributor

Nah, I suggested the wrong fix at the first place anyway :)

CodingCat pushed a commit to CodingCat/spark that referenced this pull request Aug 17, 2015
I made a mistake in apache#8049 by casting literal value to attribute's data type, which would cause simply truncate the literal value and push a wrong filter down.

JIRA: https://issues.apache.org/jira/browse/SPARK-9927

Author: Yijie Shen <[email protected]>

Closes apache#8157 from yjshen/rever8049.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants