Skip to content

Commit 6a6075a

Browse files
LantaoJinsrowen
LantaoJin
authored and
srowen
committed
[SPARK-27157][DOCS] Add Executor level metrics to monitoring docs
## What changes were proposed in this pull request? A sub-task of [SPARK-23206](https://issues.apache.org/jira/browse/SPARK-23206) Add Executor level metrics to monitoring docs ## How was this patch tested? jekyll Closes #24090 from LantaoJin/SPARK-27157. Authored-by: Lantao Jin <[email protected]> Signed-off-by: Sean Owen <[email protected]>
1 parent aea9a57 commit 6a6075a

File tree

1 file changed

+143
-0
lines changed

1 file changed

+143
-0
lines changed

‎docs/monitoring.md‎

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -609,7 +609,150 @@ A list of the available metrics, with a short description:
609609
</tr>
610610
</table>
611611

612+
### Executor Metrics
612613

614+
Executor-level metrics are sent from each executor to the driver as part of the Heartbeat to describe the performance metrics of Executor itself like JVM heap memory, GC infomation. Metrics `peakExecutorMetrics.*` are only enabled if `spark.eventLog.logStageExecutorMetrics.enabled` is true.
615+
A list of the available metrics, with a short description:
616+
617+
<table class="table">
618+
<tr><th>Executor Level Metric name</th>
619+
<th>Short description</th>
620+
</tr>
621+
<tr>
622+
<td>totalGCTime</td>
623+
<td>Elapsed time the JVM spent in garbage collection summed in this Executor.
624+
The value is expressed in milliseconds.</td>
625+
</tr>
626+
<tr>
627+
<td>totalInputBytes</td>
628+
<td>Total input bytes summed in this Executor.</td>
629+
</tr>
630+
<tr>
631+
<td>totalShuffleRead</td>
632+
<td>Total shuffer read bytes summed in this Executor.</td>
633+
</tr>
634+
<tr>
635+
<td>totalShuffleWrite</td>
636+
<td>Total shuffer write bytes summed in this Executor.</td>
637+
</tr>
638+
<tr>
639+
<td>maxMemory</td>
640+
<td>Total amount of memory available for storage, in bytes.</td>
641+
</tr>
642+
<tr>
643+
<td>memoryMetrics.*</td>
644+
<td>Current value of memory metrics:</td>
645+
</tr>
646+
<tr>
647+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.usedOnHeapStorageMemory</td>
648+
<td>Used on heap memory currently for storage, in bytes.</td>
649+
</tr>
650+
<tr>
651+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.usedOffHeapStorageMemory</td>
652+
<td>Used off heap memory currently for storage, in bytes.</td>
653+
</tr>
654+
<tr>
655+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.totalOnHeapStorageMemory</td>
656+
<td>Total available on heap memory for storage, in bytes. This amount can vary over time, on the MemoryManager implementation.</td>
657+
</tr>
658+
<tr>
659+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.totalOffHeapStorageMemory</td>
660+
<td>Total available off heap memory for storage, in bytes. This amount can vary over time, depending on the MemoryManager implementation.</td>
661+
</tr>
662+
<tr>
663+
<td>peakMemoryMetrics.*</td>
664+
<td>Peak value of memory (and GC) metrics:</td>
665+
</tr>
666+
<tr>
667+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.JVMHeapMemory</td>
668+
<td>Peak memory usage of the heap that is used for object allocation.
669+
The heap consists of one or more memory pools. The used and committed size of the returned memory usage is the sum of those values of all heap memory pools whereas the init and max size of the returned memory usage represents the setting of the heap memory which may not be the sum of those of all heap memory pools.
670+
The amount of used memory in the returned memory usage is the amount of memory occupied by both live objects and garbage objects that have not been collected, if any.</td>
671+
</tr>
672+
<tr>
673+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.JVMOffHeapMemory</td>
674+
<td>Peak memory usage of non-heap memory that is used by the Java virtual machine. The non-heap memory consists of one or more memory pools. The used and committed size of the returned memory usage is the sum of those values of all non-heap memory pools whereas the init and max size of the returned memory usage represents the setting of the non-heap memory which may not be the sum of those of all non-heap memory pools.</td>
675+
</tr>
676+
<tr>
677+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.OnHeapExecutionMemory</td>
678+
<td>Peak on heap execution memory in use, in bytes.</td>
679+
</tr>
680+
<tr>
681+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.OffHeapExecutionMemory</td>
682+
<td>Peak off heap execution memory in use, in bytes.</td>
683+
</tr>
684+
<tr>
685+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.OnHeapStorageMemory</td>
686+
<td>Peak on heap storage memory in use, in bytes.</td>
687+
</tr>
688+
<tr>
689+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.OffHeapStorageMemory</td>
690+
<td>Peak off heap storage memory in use, in bytes.</td>
691+
</tr>
692+
<tr>
693+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.OnHeapUnifiedMemory</td>
694+
<td>Peak on heap memory (execution and storage).</td>
695+
</tr>
696+
<tr>
697+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.OffHeapUnifiedMemory</td>
698+
<td>Peak off heap memory (execution and storage).</td>
699+
</tr>
700+
<tr>
701+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.DirectPoolMemory</td>
702+
<td>Peak memory that the JVM is using for direct buffer pool ([[java.lang.management.BufferPoolMXBean]])</td>
703+
</tr>
704+
<tr>
705+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.MappedPoolMemory</td>
706+
<td>Peak memory that the JVM is using for mapped buffer pool ([[java.lang.management.BufferPoolMXBean]])</td>
707+
</tr>
708+
<tr>
709+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.ProcessTreeJVMVMemory</td>
710+
<td>Virtual memory size in bytes. Enabled if spark.eventLog.logStageExecutorProcessTreeMetrics.enabled is true.</td>
711+
</tr>
712+
<tr>
713+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.ProcessTreeJVMRSSMemory</td>
714+
<td>Resident Set Size: number of pages the process has
715+
in real memory. This is just the pages which count
716+
toward text, data, or stack space. This does not
717+
include pages which have not been demand-loaded in,
718+
or which are swapped out. Enabled if spark.eventLog.logStageExecutorProcessTreeMetrics.enabled is true.</td>
719+
</tr>
720+
<tr>
721+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.ProcessTreePythonVMemory</td>
722+
<td>Virtual memory size for Python in bytes. Enabled if spark.eventLog.logStageExecutorProcessTreeMetrics.enabled is true.</td>
723+
</tr>
724+
<tr>
725+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.ProcessTreePythonRSSMemory</td>
726+
<td>Resident Set Size for Python. Enabled if spark.eventLog.logStageExecutorProcessTreeMetrics.enabled is true.</td>
727+
</tr>
728+
<tr>
729+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.ProcessTreeOtherVMemory</td>
730+
<td>Virtual memory size for other kind of process in bytes. Enabled if spark.eventLog.logStageExecutorProcessTreeMetrics.enabled is true.</td>
731+
</tr>
732+
<tr>
733+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.ProcessTreeOtherRSSMemory</td>
734+
<td>Resident Set Size for other kind of process. Enabled if spark.eventLog.logStageExecutorProcessTreeMetrics.enabled is true.</td>
735+
</tr>
736+
<tr>
737+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.MinorGCCount</td>
738+
<td>Total minor GC count. For example, the garbage collector is one of Copy, PS Scavenge, ParNew, G1 Young Generation and so on.</td>
739+
</tr>
740+
<tr>
741+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.MinorGCTime</td>
742+
<td>Elapsed total minor GC time.
743+
The value is expressed in milliseconds.</td>
744+
</tr>
745+
<tr>
746+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.MajorGCCount</td>
747+
<td>Total major GC count. For example, the garbage collector is one of MarkSweepCompact, PS MarkSweep, ConcurrentMarkSweep, G1 Old Generation and so on.</td>
748+
</tr>
749+
<tr>
750+
<td>&nbsp;&nbsp;&nbsp;&nbsp;.MajorGCTime</td>
751+
<td>Elapsed total major GC time.
752+
The value is expressed in milliseconds.</td>
753+
</tr>
754+
</table>
755+
The computation of RSS and Vmem are based on [proc(5)](http://man7.org/linux/man-pages/man5/proc.5.html)
613756

614757
### API Versioning Policy
615758

0 commit comments

Comments
 (0)