Skip to content

Conversation

@brkyvz
Copy link
Contributor

@brkyvz brkyvz commented Nov 13, 2015

Using batching on the driver for the WriteAheadLog should be an improvement for all environments and use cases. Users will be able to scale to much higher number of receivers with the BatchedWriteAheadLog. Therefore we should turn it on by default, and QA it in the QA period.

I've also added some tests to make sure the default configurations are correct regarding recent additions:

  • batching on by default
  • closeFileAfterWrite off by default
  • parallelRecovery off by default

@SparkQA
Copy link

SparkQA commented Nov 13, 2015

Test build #45872 has finished for PR 9695 at commit dfe29b8.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * public class JavaGradientBoostingClassificationExample\n * public class JavaGradientBoostingRegressionExample\n * public class JavaRandomForestClassificationExample\n * public class JavaRandomForestRegressionExample\n

@SparkQA
Copy link

SparkQA commented Nov 13, 2015

Test build #45876 has finished for PR 9695 at commit 2cbf59e.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@brkyvz
Copy link
Contributor Author

brkyvz commented Nov 13, 2015

retest this please

@SparkQA
Copy link

SparkQA commented Nov 13, 2015

Test build #45884 has finished for PR 9695 at commit 2cbf59e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@zsxwing
Copy link
Member

zsxwing commented Nov 13, 2015

LGTM

@harishreedharan
Copy link
Contributor

+1

@tdas
Copy link
Contributor

tdas commented Nov 16, 2015

LGTM. Merging this to master and 1.6

asfgit pushed a commit that referenced this pull request Nov 16, 2015
…efault

Using batching on the driver for the WriteAheadLog should be an improvement for all environments and use cases. Users will be able to scale to much higher number of receivers with the BatchedWriteAheadLog. Therefore we should turn it on by default, and QA it in the QA period.

I've also added some tests to make sure the default configurations are correct regarding recent additions:
 - batching on by default
 - closeFileAfterWrite off by default
 - parallelRecovery off by default

Author: Burak Yavuz <[email protected]>

Closes #9695 from brkyvz/enable-batch-wal.

(cherry picked from commit de5e531)
Signed-off-by: Tathagata Das <[email protected]>
@asfgit asfgit closed this in de5e531 Nov 16, 2015
@brkyvz brkyvz deleted the enable-batch-wal branch February 3, 2019 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants