Skip to content

Conversation

@HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented Oct 1, 2016

What changes were proposed in this pull request?

This PR proposes to fix/skip some tests failed on Windows. This PR takes over #12696.

Before

  • SparkSubmitSuite

    [info] - launch simple application with spark-submit *** FAILED *** (202 milliseconds)
    [info]   java.io.IOException: Cannot run program "./bin/spark-submit" (in directory "C:\projects\spark"): CreateProcess error=2, The system cannot find the file specifie
    
    [info] - includes jars passed in through --jars *** FAILED *** (1 second, 625 milliseconds)
    [info]   java.io.IOException: Cannot run program "./bin/spark-submit" (in directory "C:\projects\spark"): CreateProcess error=2, The system cannot find the file specified
    
  • DiskStoreSuite

    [info] - reads of memory-mapped and non memory-mapped files are equivalent *** FAILED *** (1 second, 78 milliseconds)
    [info]   diskStoreMapped.remove(blockId) was false (DiskStoreSuite.scala:41)
    

After

  • SparkSubmitSuite

    [info] - launch simple application with spark-submit (578 milliseconds)
    [info] - includes jars passed in through --jars (1 second, 875 milliseconds)
    
  • DiskStoreSuite

    [info] DiskStoreSuite:
    [info] - reads of memory-mapped and non memory-mapped files are equivalent !!! CANCELED !!! (766 milliseconds
    

For CreateTableAsSelectSuite and FsHistoryProviderSuite, I could not reproduce as the Java version seems higher than the one that has the bugs about setReadable(..) and setWritable(...) but as they are bugs reported clearly, it'd be sensible to skip those. We should revert the changes for both back as soon as we drop the support of Java 7.

How was this patch tested?

Manually tested via AppVeyor.

Closes #12696

@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Oct 1, 2016

cc @srowen

BTW, now I wrote some bunch of scripts and configurations for AppVeyor so I am able to run each test on Windows (except for PySpark). I will run the tests against this PR and post a comment just in case just to make sure.

@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Oct 1, 2016

So many tests seem already being failed. This PR tries to handle only the suggested changes.

Build started: [CORE] org.apache.spark.deploy.SparkSubmitSuite PR-15320

Build started: [CORE] org.apache.spark.deploy.history.FsHistoryProviderSuite PR-15320

Build started: [CORE] org.apache.spark.storage.DiskStoreSuite PR-15320

Build started: [SQL] org.apache.spark.sql.sources.CreateTableAsSelectSuite PR-15320

@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Oct 1, 2016

Sorry for being noisy, but I am thinking of sweeping the tests failed on Windows by skipping the tests across Spark (like repeating running the tests from scratch until I can't proceed further and then disabling them for each module) and then fixing them bit by bit (maybe each logically related group). It seems some PRs for Windows tests are arbitrarily being submitted. Do you think it makes sense?

@SparkQA
Copy link

SparkQA commented Oct 1, 2016

Test build #66214 has finished for PR 15320 at commit 1ff1d57.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin
Copy link
Contributor

rxin commented Oct 2, 2016

Thanks - merging in master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants