Skip to content

Conversation

@rxin
Copy link
Contributor

@rxin rxin commented Nov 18, 2015

Based on my conversions with people, I believe the consensus is that the coarse-grained mode is more stable and easier to reason about. It is best to use that as the default rather than the more flaky fine-grained mode.

@rxin
Copy link
Contributor Author

rxin commented Nov 18, 2015

cc @andrewor14 @tnachen @dragos

@dragos
Copy link
Contributor

dragos commented Nov 18, 2015

LGTM.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now that this is the default I think we can just leave this one out. Instead we should be telling people how to use fine-grained mode by setting this to false (as we already do so below)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that or say set the property to true (already the default) or something

@andrewor14
Copy link
Contributor

LGTM

@SparkQA
Copy link

SparkQA commented Nov 18, 2015

Test build #46183 has finished for PR 9795 at commit 3f5559b.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * public final class UTF8String implements Comparable<UTF8String>, Externalizable, KryoSerializable\n

@tnachen
Copy link
Contributor

tnachen commented Nov 18, 2015

LGTM as well!

@andrewor14
Copy link
Contributor

retest this please

@rxin
Copy link
Contributor Author

rxin commented Nov 18, 2015

Alright I updated the doc based on andrew's comment. Going to merge it.

asfgit pushed a commit that referenced this pull request Nov 18, 2015
Based on my conversions with people, I believe the consensus is that the coarse-grained mode is more stable and easier to reason about. It is best to use that as the default rather than the more flaky fine-grained mode.

Author: Reynold Xin <[email protected]>

Closes #9795 from rxin/SPARK-11809.

(cherry picked from commit a416e41)
Signed-off-by: Reynold Xin <[email protected]>
@asfgit asfgit closed this in a416e41 Nov 18, 2015
@SparkQA
Copy link

SparkQA commented Nov 18, 2015

Test build #46230 has finished for PR 9795 at commit 3f5559b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants