From 16c256d8206123df9487c57c6779ad2b0e0211d0 Mon Sep 17 00:00:00 2001
From: xueyu <278006819@qq.com>
Date: Sat, 23 Jun 2018 21:49:37 +0800
Subject: [PATCH 1/2] add some configs
---
docs/configuration.md | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/docs/configuration.md b/docs/configuration.md
index 6aa7878fe614d..9d1279480468f 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -456,6 +456,13 @@ Apart from these, the following properties are also available, and may be useful
from JVM to Python worker for every task.
+
+ spark.python.task.killTimeout |
+ 2s |
+
+ How long to wait before killing the python worker if a task cannot be interrupted.
+ |
+
spark.sql.repl.eagerEval.enabled |
false |
@@ -568,6 +575,13 @@ Apart from these, the following properties are also available, and may be useful
Python binary executable to use for PySpark in both driver and executors.
+
+ spark.worker.driverTerminateTimeout |
+ 10s |
+
+ Timeout to wait for when trying to terminate a driver.
+ |
+
### Shuffle Behavior
@@ -798,6 +812,13 @@ Apart from these, the following properties are also available, and may be useful
Buffer size to use when writing to output streams, in KiB unless otherwise specified.
+
+ spark.ui.consoleProgress.update.interval |
+ 200 |
+
+ Update period of progress bar, in milliseconds
+ |
+
spark.ui.enabled |
true |
From b7c9704525a113866509787c3906147fcab2882e Mon Sep 17 00:00:00 2001
From: xueyu <278006819@qq.com>
Date: Wed, 27 Jun 2018 10:50:05 +0800
Subject: [PATCH 2/2] modify config meaning
---
docs/configuration.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/configuration.md b/docs/configuration.md
index 9d1279480468f..0c1a6c9edf181 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -460,7 +460,7 @@ Apart from these, the following properties are also available, and may be useful
spark.python.task.killTimeout |
2s |
- How long to wait before killing the python worker if a task cannot be interrupted.
+ Timeout to wait before killing the python worker when a task cannot be interrupted
|