Skip to content

Commit 1445129

Browse files
cxzl25sarutak
authored andcommitted
[SPARK-33900][WEBUI] Show shuffle read size / records correctly when only remotebytesread is available
### What changes were proposed in this pull request? Shuffle Read Size / Records can also be displayed in remoteBytesRead>0 localBytesRead=0. current: ![image](https://user-images.githubusercontent.com/3898450/103079421-c4ca2280-460e-11eb-9e2f-49d35b5d324d.png) fix: ![image](https://user-images.githubusercontent.com/3898450/103079439-cc89c700-460e-11eb-9a41-6b2882980d11.png) ### Why are the changes needed? At present, the page only displays the data of Shuffle Read Size / Records when localBytesRead>0. When there is only remote reading, metrics cannot be seen on the stage page. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? manual test Closes #30916 from cxzl25/SPARK-33900. Authored-by: sychen <[email protected]> Signed-off-by: Kousuke Saruta <[email protected]> (cherry picked from commit 700f5ab) Signed-off-by: Kousuke Saruta <[email protected]>
1 parent 83adba7 commit 1445129

File tree

1 file changed

+2
-1
lines changed
  • core/src/main/resources/org/apache/spark/ui/static

1 file changed

+2
-1
lines changed

core/src/main/resources/org/apache/spark/ui/static/stagepage.js

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -821,7 +821,8 @@ $(document).ready(function () {
821821
},
822822
{
823823
data : function (row, type) {
824-
if (row.taskMetrics && row.taskMetrics.shuffleReadMetrics && row.taskMetrics.shuffleReadMetrics.localBytesRead > 0) {
824+
if (row.taskMetrics && row.taskMetrics.shuffleReadMetrics &&
825+
(row.taskMetrics.shuffleReadMetrics.localBytesRead > 0 || row.taskMetrics.shuffleReadMetrics.remoteBytesRead > 0)) {
825826
var totalBytesRead = parseInt(row.taskMetrics.shuffleReadMetrics.localBytesRead) + parseInt(row.taskMetrics.shuffleReadMetrics.remoteBytesRead);
826827
if (type === 'display') {
827828
return formatBytes(totalBytesRead, type) + " / " + row.taskMetrics.shuffleReadMetrics.recordsRead;

0 commit comments

Comments
 (0)