Skip to content

Conversation

@yhuai
Copy link
Contributor

@yhuai yhuai commented Sep 10, 2015

If hadoopFsRelationSuites's "test all data types" is too flaky we can disable it for now.

https://issues.apache.org/jira/browse/SPARK-10540

@SparkQA
Copy link

SparkQA commented Sep 10, 2015

Test build #42284 has finished for PR 8705 at commit 7c4791f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

asfgit pushed a commit that referenced this pull request Sep 11, 2015
…s" if it is too flaky

If hadoopFsRelationSuites's "test all data types" is too flaky we can disable it for now.

https://issues.apache.org/jira/browse/SPARK-10540

Author: Yin Huai <[email protected]>

Closes #8705 from yhuai/SPARK-10540-ignore.

(cherry picked from commit 6ce0886)
Signed-off-by: Yin Huai <[email protected]>
@asfgit asfgit closed this in 6ce0886 Sep 11, 2015
@liancheng
Copy link
Contributor

Found two possible sources of flakiness:

  1. For double and float, NaN == NaN is always false

  2. checkAnswer doesn't handle Map[K, V] properly. For example:

    scala> Map(1 -> 2, 2 -> 1).toString
    res0: String = Map(1 -> 2, 2 -> 1)
    
    scala> Map(2 -> 1, 1 -> 2).toString
    res1: String = Map(2 -> 1, 1 -> 2)
    

    We can't rely on toString to compare Map instances.

Need to update checkAnswer to special case NaN and Map[K, V].

@liancheng
Copy link
Contributor

Filed https://issues.apache.org/jira/browse/SPARK-10591 to track this issue.

@yhuai
Copy link
Contributor Author

yhuai commented Sep 14, 2015

For the issue of NaN, looks like there is something wrong because we want to return true when compare two NaN.

ashangit pushed a commit to ashangit/spark that referenced this pull request Oct 19, 2016
…s" if it is too flaky

If hadoopFsRelationSuites's "test all data types" is too flaky we can disable it for now.

https://issues.apache.org/jira/browse/SPARK-10540

Author: Yin Huai <[email protected]>

Closes apache#8705 from yhuai/SPARK-10540-ignore.

(cherry picked from commit 6ce0886)
Signed-off-by: Yin Huai <[email protected]>
(cherry picked from commit 295281f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants