Skip to content

Conversation

@xuanyuanking
Copy link
Member

What changes were proposed in this pull request?

Add EmptyDirectoryWriteTask for empty task while writing files.

How was this patch tested?

Add new test in FileFormatWriterSuite

@HyukjinKwon
Copy link
Member

Please refer the discussion in #17395. I am quite sure this is going to break

spark.range(100).filter("id > 100").write.parquet("/tmp/abc")
spark.read.parquet("/tmp/abc").show()

I think we should close and propose another approach in a separate PR.

@SparkQA
Copy link

SparkQA commented Jul 17, 2017

Test build #79661 has finished for PR 18650 at commit ff92ba3.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jul 17, 2017

Test build #79665 has finished for PR 18650 at commit e08fb19.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@xuanyuanking
Copy link
Member Author

Yep, just close this and open #18654

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants