Skip to content

Commit 56e746c

Browse files
author
Davies Liu
committed
preserve first page in UnsafeShuffleExternalSorter
1 parent 52ea399 commit 56e746c

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

core/src/main/java/org/apache/spark/shuffle/unsafe/UnsafeShuffleExternalSorter.java

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,10 @@ public UnsafeShuffleExternalSorter(
122122
this.maxRecordSizeBytes = pageSizeBytes - 4;
123123
this.writeMetrics = writeMetrics;
124124
initializeForWriting();
125+
126+
// preserve first page to ensure that we have at least one page to work with. Otherwise,
127+
// other operators in the same task may starve this sorter (SPARK-9709).
128+
acquireNewPageIfNecessary(pageSizeBytes);
125129
}
126130

127131
/**

0 commit comments

Comments
 (0)