Skip to content

Commit 0c00391

Browse files
Thomas GravesAndrew Or
authored andcommitted
[SPARK-15121] Improve logging of external shuffle handler
## What changes were proposed in this pull request? Add more informative logging in the external shuffle service to aid in debugging who is connecting to the YARN Nodemanager when the external shuffle service runs under it. ## How was this patch tested? Ran and saw logs coming out in log file. Author: Thomas Graves <[email protected]> Closes #12900 from tgravescs/SPARK-15121.
1 parent 6ae9fc0 commit 0c00391

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@
3535
import org.apache.spark.network.server.StreamManager;
3636
import org.apache.spark.network.shuffle.ExternalShuffleBlockResolver.AppExecId;
3737
import org.apache.spark.network.shuffle.protocol.*;
38+
import org.apache.spark.network.util.NettyUtils;
3839
import org.apache.spark.network.util.TransportConf;
3940

4041

@@ -86,7 +87,8 @@ protected void handleMessage(
8687
blocks.add(blockManager.getBlockData(msg.appId, msg.execId, blockId));
8788
}
8889
long streamId = streamManager.registerStream(client.getClientId(), blocks.iterator());
89-
logger.trace("Registered streamId {} with {} buffers", streamId, msg.blockIds.length);
90+
logger.trace("Registered streamId {} with {} buffers for client {} from host {}", streamId,
91+
msg.blockIds.length, client.getClientId(), NettyUtils.getRemoteAddress(client.getChannel()));
9092
callback.onSuccess(new StreamHandle(streamId, msg.blockIds.length).toByteBuffer());
9193

9294
} else if (msgObj instanceof RegisterExecutor) {

0 commit comments

Comments
 (0)