Skip to content

Commit 7feae6e

Browse files
committed
Add spark.yarn.dist.jars to doc and fix minor issue
1 parent d152f9f commit 7feae6e

File tree

3 files changed

+12
-6
lines changed

3 files changed

+12
-6
lines changed

docs/running-on-yarn.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -215,6 +215,13 @@ If you need a reference to the proper location to put log files in the YARN so t
215215
Comma-separated list of files to be placed in the working directory of each executor.
216216
</td>
217217
</tr>
218+
<tr>
219+
<td><code>spark.yarn.dist.jars</code></td>
220+
<td>(none)</td>
221+
<td>
222+
Comma-separated list of jars to be placed in the working directory of each executor.
223+
</td>
224+
</tr>
218225
<tr>
219226
<td><code>spark.executor.cores</code></td>
220227
<td>1 in YARN mode, all the available cores on the worker in standalone mode.</td>

yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMasterArguments.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ class ApplicationMasterArguments(val args: Array[String]) {
9191
| --class CLASS_NAME Name of your application's main class
9292
| --primary-py-file A main Python file
9393
| --primary-r-file A main R file
94-
| --args ARGS Arguments to be passed to your application's main class.
94+
| --arg ARG Argument to be passed to your application's main class.
9595
| Multiple invocations are possible, each will be passed in order.
9696
| --properties-file FILE Path to a custom Spark properties file.
9797
""".stripMargin)

yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,10 @@ package object config {
9191
.stringConf
9292
.optional
9393

94+
private[spark] val JARS_TO_DISTRIBUTE = ConfigBuilder("spark.yarn.dist.jars")
95+
.stringConf
96+
.optional
97+
9498
private[spark] val PRESERVE_STAGING_FILES = ConfigBuilder("spark.yarn.preserve.staging.files")
9599
.doc("Whether to preserve temporary files created by the job in HDFS.")
96100
.booleanConf
@@ -249,9 +253,4 @@ package object config {
249253
.stringConf
250254
.toSequence
251255
.optional
252-
253-
private[spark] val JARS_TO_DISTRIBUTE = ConfigBuilder("spark.yarn.dist.jars")
254-
.internal
255-
.stringConf
256-
.optional
257256
}

0 commit comments

Comments
 (0)